Nov 02 14:53:46.494139 ip-10-0-139-197 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 02 14:53:46.494150 ip-10-0-139-197 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 02 14:53:46.494160 ip-10-0-139-197 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 02 14:53:46.494530 ip-10-0-139-197 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 02 14:53:56.738809 ip-10-0-139-197 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 02 14:53:56.738827 ip-10-0-139-197 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 87227f92df8e4e919175f17f34b5e512 --
Nov 02 14:55:52.095773 ip-10-0-139-197 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 14:55:52.484339 ip-10-0-139-197 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:55:52.484339 ip-10-0-139-197 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 14:55:52.484339 ip-10-0-139-197 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:55:52.484339 ip-10-0-139-197 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:55:52.488587 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488509    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.488587 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488586    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.488587 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488591    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488594    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488597    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488601    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488603    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488606    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488609    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488611    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488614    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488617    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488619    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488622    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488626    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488630    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488634    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488638    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488641    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488644    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488647    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.488686 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488650    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488652    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488655    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488663    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488666    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488669    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488672    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488675    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488677    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488680    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488683    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488687    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488691    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488695    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488698    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488700    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488703    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488706    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488709    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.489120 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488711    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488714    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488716    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488719    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488721    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488724    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488727    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488729    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488732    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488736    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488739    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488741    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488744    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488746    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488748    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488751    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488754    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488757    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488760    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488762    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.489606 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488765    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488768    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488771    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488773    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488777    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488780    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488782    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488785    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488788    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488790    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488793    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488795    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488798    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488800    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488803    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488805    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488808    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488810    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488813    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488815    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.490080 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488818    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488821    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488824    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488827    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488829    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488832    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488835    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488837    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488840    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488843    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488845    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488848    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488851    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488854    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488857    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488860    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488863    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488866    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488868    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488871    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.490561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488874    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488876    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488879    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488881    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488884    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.488886    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490344    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490353    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490356    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490359    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490362    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490365    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490368    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490371    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490374    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490377    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490379    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490382    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490385    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.491036 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490387    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490390    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490392    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490395    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490397    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490401    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490403    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490406    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490410    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490414    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490417    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490420    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490423    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490425    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490428    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490431    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490434    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490437    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490440    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.491502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490442    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490445    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490448    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490450    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490454    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490459    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490463    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490466    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490481    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490486    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490489    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490493    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490496    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490498    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490501    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490504    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490506    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490509    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490512    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490514    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.491974 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490517    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490520    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490523    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490525    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490528    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490530    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490533    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490536    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490540    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490542    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490545    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490548    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490550    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490553    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490556    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490558    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490561    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490564    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490567    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490570    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.492462 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490573    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490576    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490579    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490582    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490585    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490588    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490590    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490593    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490596    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490598    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490601    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490603    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490606    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490609    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490611    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490614    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490616    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490619    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490621    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.492948 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490624    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490627    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490629    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490631    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490634    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490636    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490639    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490642    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490645    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490649    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490652    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490655    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490657    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490659    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.490662    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491157    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491163    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491166    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491168    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491171    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.493431 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491174    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491177    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491180    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491183    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491185    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491188    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491191    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491194    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491196    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491199    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491201    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491204    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491206    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491209    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491211    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491214    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491216    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491219    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491222    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.493915 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491224    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491227    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491230    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491232    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491235    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491238    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491241    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491244    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491246    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491249    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491252    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491255    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491260    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491264    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491267    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491270    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491273    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491275    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491278    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.494360 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491280    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491283    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491285    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491288    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491290    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491293    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491295    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491298    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491301    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491303    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491306    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491308    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491311    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491313    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491316    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491321    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491324    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491326    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491329    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491331    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.494861 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491334    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491336    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491339    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491342    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491344    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491347    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491349    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491352    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491354    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491357    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491360    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491363    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491365    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491367    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491370    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491373    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491375    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491378    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491380    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491382    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.495336 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491385    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491387    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491390    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491392    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491395    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491399    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491402    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491406    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491409    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491412    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491414    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491417    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491420    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491422    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491425    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491427    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491430    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491433    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491435    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.495846 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491439    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491441    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491444    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.491446    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491551    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491563    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491573    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491583    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491590    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491594    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491598    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491603    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491606    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491609    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491613    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491616    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491619    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491623    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491626    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491629    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491635    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491639    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491644    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491647    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 14:55:52.496333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491650    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491653    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491657    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491661    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491664    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491667    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491671    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491674    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491677    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491681    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491684    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491687    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491691    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491694    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491697    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491702    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491705    2580 flags.go:64] FLAG: --enable-server="true"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491708    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491712    2580 flags.go:64] FLAG: --event-burst="100"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491716    2580 flags.go:64] FLAG: --event-qps="50"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491719    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491722    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491725    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491729    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 14:55:52.496905 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491733    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491736    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491739    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491742    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491747    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491750    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491754    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491757    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491760    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491763    2580 flags.go:64] FLAG: --feature-gates=""
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491767    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491770    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491774    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491780    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491783    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491787    2580 flags.go:64] FLAG: --help="false"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491790    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491794    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491797    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491799    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491803    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491806    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491809    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 14:55:52.497521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491814    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491817    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491820    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491823    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491826    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491829    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491832    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491834    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491838    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491840    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491843    2580 flags.go:64] FLAG: --lock-file=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491847    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491850    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491854    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491860    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491863    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491866    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491869    2580 flags.go:64] FLAG: --logging-format="text"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491872    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491876    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491879    2580 flags.go:64] FLAG: --manifest-url=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491882    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491886    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491889    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491893    2580 flags.go:64] FLAG: --max-pods="110"
Nov 02 14:55:52.498174 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491897    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491907    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491911    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491914    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491917    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491920    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491923    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491932    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491935    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491938    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491942    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491945    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491948    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491951    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491954    2580 flags.go:64] FLAG: --port="10250"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491957    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491960    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-03d898ffef658186f"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491963    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491967    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491970    2580 flags.go:64] FLAG: --register-node="true"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491974    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491978    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491981    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491984    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491987    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491992    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 14:55:52.498811 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491995    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.491998    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492001    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492004    2580 flags.go:64] FLAG: --runonce="false"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492007    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492010    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492013    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492017    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492020    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492023    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492027    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492030    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492033    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492037    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492040    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492043    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492046    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492049    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492052    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492057    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492060    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492064    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492069    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492072    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492074    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 14:55:52.499426 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492077    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492081    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492084    2580 flags.go:64] FLAG: --v="2"
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492092    2580 flags.go:64] FLAG: --version="false"
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492097    2580 flags.go:64] FLAG: --vmodule=""
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492100    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492104    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492255    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492259    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492263    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492267    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492270    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492273    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492276    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492278    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492281    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492284    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492288    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492292    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492295    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.500073 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492299    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492301    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492304    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492306    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492309    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492312    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492314    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492317    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492319    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492322    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492324    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492327    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492330    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492333    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492335    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492338    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492340    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492343    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492346    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492348    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.500555 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492351    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492353    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492356    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492358    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492361    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492363    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492366    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492368    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492371    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492374    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492376    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492378    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492382    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492385    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492387    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492390    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492392    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492395    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492397    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492400    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.501034 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492402    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492404    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492407    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492410    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492412    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492415    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492418    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492420    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492423    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492425    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492428    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492430    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492433    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492436    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492438    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492441    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492443    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492446    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492448    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.501683 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492451    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492453    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492456    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492459    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492461    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492465    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492482    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492486    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492491    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492495    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492498    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492501    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492503    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492506    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492508    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492511    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492514    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492516    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492519    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492521    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.502260 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492524    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492527    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492530    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492532    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492534    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492537    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492539    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492542    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492544    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492547    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492549    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492552    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492554    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492556    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.492585    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492625    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.502751 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492631    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492636    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492640    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492643    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492647    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492650    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492655    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492660    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492662    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492665    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492669    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492673    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492676    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492680    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492682    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492685    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492688    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492691    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492694    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.503130 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492696    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492699    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492701    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492704    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492707    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492710    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492713    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492715    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492718    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492721    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492723    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492725    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492728    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492731    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492734    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492736    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492739    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492741    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492744    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492746    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.503611 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492750    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492753    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492755    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492758    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492761    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492763    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492766    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492769    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492771    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492773    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492776    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492779    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492781    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492784    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492786    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492789    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492792    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492794    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492797    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.504095 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492799    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492802    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492804    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492807    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492809    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492812    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492814    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492817    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492820    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492823    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492825    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492828    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492830    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492834    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492837    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492839    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492841    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492844    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492846    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.504557 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492849    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492852    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492854    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492857    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492859    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492862    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492864    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492867    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492869    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492872    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492874    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492877    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492880    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492882    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492885    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492888    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492891    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492893    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492896    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492898    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.505012 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492901    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492904    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492906    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492909    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492912    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492914    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492918    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.492920    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499128    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499143    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499154    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499159    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499218    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499223    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499226    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499230    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499233    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499237    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499241    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.505625 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499244    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499248    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499250    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499253    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499256    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499258    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499261    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499264    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499267    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499269    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499272    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499275    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499277    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499280    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499283    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499286    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499288    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499291    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499293    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.506076 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499297    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499299    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499302    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499305    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499307    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499310    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499312    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499315    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499317    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499320    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499323    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499326    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499329    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499331    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499334    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499336    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499339    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499341    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499344    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499346    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.506550 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499349    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499351    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499354    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499356    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499359    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499361    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499363    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499366    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499369    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499372    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499375    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499377    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499380    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499382    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499385    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499388    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499390    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499393    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499395    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.507017 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499398    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499400    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499403    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499405    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499408    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499410    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499413    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499415    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499418    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499420    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499423    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499425    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499428    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499431    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499433    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499436    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499439    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499442    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499444    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.507467 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499447    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499449    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499452    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499455    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499458    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499460    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499463    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499465    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499479    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499482    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499484    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499487    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499489    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499492    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499495    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499497    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499500    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499502    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499505    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.507935 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499509    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499512    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499515    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499542    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499677    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499682    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499685    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499688    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499690    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499693    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499696    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499698    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499701    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499704    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499707    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.508407 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499712    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499715    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499721    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499724    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499727    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499730    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499732    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499735    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499737    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499740    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499743    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499745    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499748    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499751    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499753    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499756    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499758    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499761    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499763    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499766    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.508822 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499768    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499771    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499773    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499776    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499778    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499781    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499783    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499786    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499788    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499791    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499793    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499796    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499798    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499801    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499803    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499806    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499809    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499812    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499814    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499817    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.509274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499819    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499822    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499824    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499827    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499830    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499832    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499835    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499837    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499840    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499842    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499845    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499847    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499850    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499852    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499854    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499857    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499859    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499862    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499864    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499867    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.509760 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499869    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499872    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499874    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499877    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499879    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499882    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499884    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499889    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499893    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499896    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499906    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499909    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499912    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499915    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499917    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499920    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499922    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499925    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499928    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.510232 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499930    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499933    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499936    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499938    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499942    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499944    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499947    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499950    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499952    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499955    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499959    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499962    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499966    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499968    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499971    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.510766 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.499974    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.499996    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500029    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500034    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500037    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500042    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500044    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500049    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500052    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500055    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500058    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500060    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500063    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500065    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500068    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500071    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:55:52.511113 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500073    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500076    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500079    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500082    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500085    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500087    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500090    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500092    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500095    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500109    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500113    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500117    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500120    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500123    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500126    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500128    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500131    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500133    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500136    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500138    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:55:52.511506 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500141    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500145    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500147    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500151    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500154    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500157    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500160    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500163    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500166    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500168    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500171    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500175    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500179    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500182    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500185    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500188    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500191    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500194    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500197    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:55:52.511989 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500200    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500202    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500205    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500208    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500210    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500213    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500216    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500218    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500222    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500225    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500228    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500231    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500234    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500236    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500241    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500243    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500247    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500250    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500254    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:55:52.512480 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500257    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500260    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500263    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500265    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500268    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500270    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500273    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500276    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500278    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500281    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500284    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500286    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500289    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500291    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500294    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500296    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500299    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500301    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500304    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:55:52.512924 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500307    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500309    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500312    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500314    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500317    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500319    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500322    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500324    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500329    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500331    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500337    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500340    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500342    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500346    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:52.500349    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.500398    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.503844    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.504843    2580 server.go:1014] "Starting client certificate rotation"
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.504937    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 14:55:52.513374 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.504978    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 14:55:52.513850 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.506883    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 14:55:52.513850 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.508045    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 14:55:52.518362 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.518349    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 02 14:55:52.522624 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.522611    2580 log.go:25] "Validated CRI v1 image API"
Nov 02 14:55:52.523366 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.523345    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 14:55:52.526249 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.526230    2580 fs.go:135] Filesystem UUIDs: map[2cfa3683-7091-4562-81cc-36493a3ce05b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c7eb937a-6db3-4ff7-ae22-4aaef4a89837:/dev/nvme0n1p3]
Nov 02 14:55:52.526316 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.526247    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 14:55:52.532876 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.532756    2580 manager.go:217] Machine: {Timestamp:2026-11-02 14:55:52.531499183 +0000 UTC m=+0.323145795 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098787 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b4cee81dff1443501dfc355863f1d SystemUUID:ec2b4cee-81df-f144-3501-dfc355863f1d BootID:87227f92-df8e-4e91-9175-f17f34b5e512 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:13:10:c9:82:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:13:10:c9:82:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:7b:d5:3f:dc:b5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 14:55:52.533458 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.533449    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 14:55:52.533584 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.533567    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 14:55:52.534559 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.534537    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 14:55:52.534729 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.534562    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-197.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 14:55:52.535225 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.535213    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 14:55:52.535280 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.535230    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 14:55:52.535280 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.535267    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 14:55:52.535355 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.535287    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 14:55:52.535948 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.535935    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 14:55:52.539755 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.539741    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 14:55:52.539891 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.539880    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 14:55:52.540491 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540466    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 14:55:52.540544 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540499    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 14:55:52.540544 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540513    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 14:55:52.540544 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540527    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 02 14:55:52.540544 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540538    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 14:55:52.540940 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.540927    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 14:55:52.543994 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.543973    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 14:55:52.547816 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.547803    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 14:55:52.548501 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548489    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 14:55:52.548538 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548526    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 14:55:52.548711 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548700    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 14:55:52.548745 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548716    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 14:55:52.548745 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548722    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 14:55:52.548745 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548727    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 14:55:52.548745 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548744    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 14:55:52.548851 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.548754    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 14:55:52.549257 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549247    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 14:55:52.549286 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549258    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 14:55:52.549286 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549265    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 14:55:52.549286 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549271    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 14:55:52.549286 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549279    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 14:55:52.549406 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549288    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 14:55:52.549774 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549764    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 14:55:52.549810 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.549777    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 14:55:52.553324 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.553310    2580 server.go:1263] "Started kubelet"
Nov 02 14:55:52.553451 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.553423    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 14:55:52.553451 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.553414    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 14:55:52.553593 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.553488    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 14:55:52.554092 ip-10-0-139-197 systemd[1]: Started Kubernetes Kubelet.
Nov 02 14:55:52.554514 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.554496    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 14:55:52.558432 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.558369    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 14:55:52.558649 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.558633    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 14:55:52.558714 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.558651    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 14:55:52.558829 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.558814    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:55:52.559363 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.559342    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 14:55:52.559442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.559365    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 14:55:52.559442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.559406    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 14:55:52.559586 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.559567    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 14:55:52.559586 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.559583    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 14:55:52.560865 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.560843    2580 factory.go:55] Registering systemd factory
Nov 02 14:55:52.560865 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.560862    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 02 14:55:52.561001 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.560910    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 14:55:52.561109 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.561090    2580 factory.go:153] Registering CRI-O factory
Nov 02 14:55:52.561109 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.561109    2580 factory.go:223] Registration of the crio container factory successfully
Nov 02 14:55:52.561229 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.561159    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 14:55:52.561229 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.561176    2580 factory.go:103] Registering Raw factory
Nov 02 14:55:52.561229 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.561189    2580 manager.go:1199] Started watching for new ooms in manager
Nov 02 14:55:52.562064 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.562051    2580 manager.go:319] Starting recovery of all containers
Nov 02 14:55:52.572270 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.572249    2580 manager.go:324] Recovery completed
Nov 02 14:55:52.579004 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.578987    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.581027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581008    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.581096 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581045    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.581096 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581059    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.581694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581678    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 02 14:55:52.581694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581688    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.581694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581693    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 14:55:52.581895 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.581715    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 14:55:52.582448 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.582424    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.582518 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.582452    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.582518 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.582461    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.584031 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.584014    2580 policy_none.go:50] "Start"
Nov 02 14:55:52.584094 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.584036    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 14:55:52.584094 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.584051    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 14:55:52.586159 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.586146    2580 policy_none.go:44] "Start"
Nov 02 14:55:52.624033 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.624004    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 14:55:52.627962 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.627943    2580 manager.go:341] "Starting Device Plugin manager"
Nov 02 14:55:52.628459 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.628440    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 14:55:52.628511 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.628464    2580 server.go:87] "Starting device plugin registration server"
Nov 02 14:55:52.628776 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.628761    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 14:55:52.629027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.628779    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 14:55:52.629027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.628882    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 14:55:52.629027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.629004    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 14:55:52.629027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.629014    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 14:55:52.630648 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.630574    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 14:55:52.630648 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.630631    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:55:52.691440 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.691403    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 14:55:52.693398 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.692601    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 14:55:52.693398 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.692620    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 14:55:52.693398 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.692640    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 14:55:52.693398 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.692678    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 02 14:55:52.702897 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.702864    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-197.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 14:55:52.702991 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.702904    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-197.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 14:55:52.729528 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.729507    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.730327 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.730312    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.730416 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.730345    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.730416 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.730361    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.730416 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.730393    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.751918 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.751055    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-197.us-west-1.compute.internal.18e442364af1258b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-197.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-197.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 14:55:52.553285003 +0000 UTC m=+0.344931617,LastTimestamp:2026-11-02 14:55:52.553285003 +0000 UTC m=+0.344931617,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-197.us-west-1.compute.internal,}"
Nov 02 14:55:52.781669 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.781647    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 14:55:52.781786 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.781707    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-czf64"
Nov 02 14:55:52.793537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.793501    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"]
Nov 02 14:55:52.793607 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.793576    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.798527 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.798510    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.798618 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.798546    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.798618 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.798564    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.800761 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.800744    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.800890 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.800875    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.800940 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.800905    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.801624 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801609    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.801692 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801638    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.801692 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801648    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.801763 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801614    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.801763 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801714    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.801763 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.801725    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.803712 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.803697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.803785 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.803726    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:55:52.804515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.804495    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:55:52.804586 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.804531    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:55:52.804586 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.804546    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:55:52.826827 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.826809    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.829214 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.829199    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.861251 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.861222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.861410 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.861257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.861410 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.861299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.864734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.864698    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:52.866599 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.866586    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:52.962081 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.962081 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.962262 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.962262 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962136    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.962262 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962142    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.962262 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.962177    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.965974 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.965953    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:52.967578 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.967562    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:52.967728 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:52.967714    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:52.967790 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:52.967739    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-197.us-west-1.compute.internal\": node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:55:53.059359 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.059282    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:53.126302 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.126277    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 14:55:53.126447 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.126388    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:53.129447 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.129419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:53.132478 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.132457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:55:53.183785 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:53.183618    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 14:55:54.971795 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:54.971766    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 14:55:54.972312 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:54.971958    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 14:55:54.972312 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:54.971958    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 14:55:54.972312 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:54.971929    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a65af20ceb8e0453d8b17c9c993c7f5c-df77204d0ad5c075.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-139-197.us-west-1.compute.internal.18e442364c99093f\": read tcp 10.0.139.197:48748->3.135.56.126:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-139-197.us-west-1.compute.internal.18e442364c99093f  default   5054 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-197.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-197.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-197.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 14:55:52 +0000 UTC,LastTimestamp:2026-11-02 14:55:52.730367317 +0000 UTC m=+0.522013934,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-197.us-west-1.compute.internal,}"
Nov 02 14:55:54.972312 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:54.971962    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 02 14:55:55.007814 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.007788    2580 apiserver.go:52] "Watching apiserver"
Nov 02 14:55:55.025521 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.025489    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 14:55:55.253468 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.253419    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 14:50:54 +0000 UTC" deadline="2028-04-14 03:38:29.304403366 +0000 UTC"
Nov 02 14:55:55.253468 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.253459    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12684h42m34.050949395s"
Nov 02 14:55:55.284052 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.284017    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 14:55:55.284398 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.284370    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-xrvk7","openshift-multus/network-metrics-daemon-z6trp","kube-system/global-pull-secret-syncer-d7qbm","openshift-multus/multus-additional-cni-plugins-hl98t","openshift-multus/multus-d59wq","openshift-network-diagnostics/network-check-target-g5d86","openshift-image-registry/node-ca-ldxkl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal","openshift-ovn-kubernetes/ovnkube-node-kndjb","kube-system/konnectivity-agent-vsttp","kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-qs4d5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74","openshift-cluster-node-tuning-operator/tuned-sz7rm"]
Nov 02 14:55:55.286933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.286918    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.288961 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.288939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.289113 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.289005    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:55:55.290983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.290967    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.291107 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.291090    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:55:55.293049 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.293037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.295117 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.295100    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.297130 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.297117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:55.297207 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.297162    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:55:55.299252 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.299239    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.301569 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.301545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.303693 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.303676    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.308563 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.308548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.311161 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.311147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.312945 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.312930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.326999 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.326984    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 14:55:55.332494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.332549 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.332549 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.332549 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.332636 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.332636 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.332694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.332694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztmd9\" (UniqueName: \"kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.332694 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.332771 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.332771 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.332771 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.332867 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.332867 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.332867 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332812    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.332867 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.332980 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.332980 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.332980 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332930    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.332980 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8z7\" (UniqueName: \"kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.332990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvdz5\" (UniqueName: \"kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.333093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333086    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x7t6\" (UniqueName: \"kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333178    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd7ww\" (UniqueName: \"kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.333277 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvz6c\" (UniqueName: \"kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333557    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333692    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.333734 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.333954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.334120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxfqw\" (UniqueName: \"kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334329    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts97h\" (UniqueName: \"kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zpb8\" (UniqueName: \"kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.334496 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.334983 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.334931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.339429 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.338890    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.339429 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.338900    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rrdxh\""
Nov 02 14:55:55.339429 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.339010    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.344614 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344596    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.344710 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344662    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8dn6m\""
Nov 02 14:55:55.344748 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344718    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 14:55:55.344748 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344727    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 14:55:55.344800 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344785    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 14:55:55.344916 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.344902    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.346383 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.346371    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-5w9z9\""
Nov 02 14:55:55.346495 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.346483    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 14:55:55.351046 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.351031    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 14:55:55.351131 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.351060    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.351131 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.351094    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m4lkk\""
Nov 02 14:55:55.351131 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.351104    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.353044 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.353025    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5lvgl\""
Nov 02 14:55:55.436077 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436044    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8z7\" (UniqueName: \"kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvdz5\" (UniqueName: \"kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.436240 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436220    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436219    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436259    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436266    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436489    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7t6\" (UniqueName: \"kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436518    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 14:55:55.436537 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436581    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd7ww\" (UniqueName: \"kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436646    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6wvcv"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436678    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436865    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.436933 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436948    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436966    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.436994    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437005    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvz6c\" (UniqueName: \"kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437032    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437120    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437169    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.437515 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437213    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437249    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437292    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437326    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437343    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437354    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437414    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437417    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.438365 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437488    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437546    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437622    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437651    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxfqw\" (UniqueName: \"kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437608    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437667    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437701    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437705    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts97h\" (UniqueName: \"kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437745    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zpb8\" (UniqueName: \"kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.439178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437945    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437948    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437947    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.437991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438023    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438064    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438159    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.439740 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438219    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438212    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.438327    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438108    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438400    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438407    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.438430    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:55.93839127 +0000 UTC m=+2.263320590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438481    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.440494 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438586    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438562    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438515    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438625    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztmd9\" (UniqueName: \"kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438690    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.438701    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438720    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.438763    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:55.938747068 +0000 UTC m=+2.263676393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438791    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438856    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.441333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.438971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439011    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439018    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439012    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439054    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439066    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439086    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439103    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439581    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439740    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439848    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439876    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439920    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439930    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439878    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439900    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439892    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439963    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.442160 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.439961    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440102    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440125    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440156    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dhf26\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440227    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440241    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zp9vx\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440280    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zct6j\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440334    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440430    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440468    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440609    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-snsjj\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440644    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 14:55:55.443253 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.440739    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 14:55:55.447427 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.447409    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.448357 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.448335    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.448459 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.448343    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.449627 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.449604    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.450472 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.450453    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.450595 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.450581    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.450637 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.450582    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.494809 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.494778    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:55:55.494943 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.494829    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:55:55.494943 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.494844    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:55.495024 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.494945    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:55.994916059 +0000 UTC m=+2.319845372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:55.497533 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.497504    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd7ww\" (UniqueName: \"kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.497645 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.497507    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7t6\" (UniqueName: \"kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.497645 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.497628    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvz6c\" (UniqueName: \"kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.497822 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.497802    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvdz5\" (UniqueName: \"kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.497855 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.497804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8z7\" (UniqueName: \"kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.498342 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.498326    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts97h\" (UniqueName: \"kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.501919 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.501895    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztmd9\" (UniqueName: \"kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.502101 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.502076    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxfqw\" (UniqueName: \"kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.502565 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.502547    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zpb8\" (UniqueName: \"kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.621636 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.621557    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:55:55.628631 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.628602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:55:55.654169 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.654135    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:55:55.657285 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.657257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d59wq"
Nov 02 14:55:55.666979 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.666957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:55:55.697238 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.697209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:55:55.697744 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.697705    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0c7197_ef21_4d95_94bc_133fbd54c166.slice/crio-b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1 WatchSource:0}: Error finding container b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1: Status 404 returned error can't find the container with id b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1
Nov 02 14:55:55.698033 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.698002    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7f87aa5e9c9e37d740f6ee1d873224.slice/crio-551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c WatchSource:0}: Error finding container 551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c: Status 404 returned error can't find the container with id 551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c
Nov 02 14:55:55.698274 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.698235    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd479be79ef05ee5c1afc91f51096ca7.slice/crio-3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed WatchSource:0}: Error finding container 3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed: Status 404 returned error can't find the container with id 3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed
Nov 02 14:55:55.702068 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.702039    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:55:55.702771 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.702754    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 14:55:55.709270 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.709246    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e573dd1_b81f_4134_9454_739323385c26.slice/crio-4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc WatchSource:0}: Error finding container 4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc: Status 404 returned error can't find the container with id 4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc
Nov 02 14:55:55.712025 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.712005    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84352a64_ea29_427b_a1bb_93bd7aa3298a.slice/crio-ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46 WatchSource:0}: Error finding container ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46: Status 404 returned error can't find the container with id ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46
Nov 02 14:55:55.723250 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.723225    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:55:55.725340 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.725323    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:55:55.733807 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.733788    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3b4ac6_1c86_4cff_88e6_b3789fc6fe95.slice/crio-22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5 WatchSource:0}: Error finding container 22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5: Status 404 returned error can't find the container with id 22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5
Nov 02 14:55:55.734727 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:55.734706    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cac903e_68cd_473b_9177_e0acbcbee02a.slice/crio-26133ef274ce020396451332121441922f17e863944e95ead042deeef8c287f9 WatchSource:0}: Error finding container 26133ef274ce020396451332121441922f17e863944e95ead042deeef8c287f9: Status 404 returned error can't find the container with id 26133ef274ce020396451332121441922f17e863944e95ead042deeef8c287f9
Nov 02 14:55:55.942988 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.942952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:55.943177 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.943002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:55.943177 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.943109    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:55.943177 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.943172    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:56.943155136 +0000 UTC m=+3.268084474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:55.943342 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.943109    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:55.943342 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:55.943225    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:56.943219047 +0000 UTC m=+3.268148354 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:55.964518 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:55.964481    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:56.043282 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.043246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:56.043947 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.043441    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:55:56.043947 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.043466    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:55:56.043947 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.043478    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:56.043947 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.043549    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:57.043529663 +0000 UTC m=+3.368458977 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:56.163377 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.163331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ldxkl" event={"ID":"bd0c7197-ef21-4d95-94bc-133fbd54c166","Type":"ContainerStarted","Data":"b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1"}
Nov 02 14:55:56.164367 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.164343    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerStarted","Data":"3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed"}
Nov 02 14:55:56.165442 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.165420    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sz7rm" event={"ID":"8cac903e-68cd-473b-9177-e0acbcbee02a","Type":"ContainerStarted","Data":"26133ef274ce020396451332121441922f17e863944e95ead042deeef8c287f9"}
Nov 02 14:55:56.166467 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.166449    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74" event={"ID":"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95","Type":"ContainerStarted","Data":"22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5"}
Nov 02 14:55:56.167509 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.167487    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs4d5" event={"ID":"84352a64-ea29-427b-a1bb-93bd7aa3298a","Type":"ContainerStarted","Data":"ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46"}
Nov 02 14:55:56.168574 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.168557    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc"}
Nov 02 14:55:56.169629 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.169609    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"6e7f87aa5e9c9e37d740f6ee1d873224","Type":"ContainerStarted","Data":"551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c"}
Nov 02 14:55:56.309483 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.309404    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:56.440218 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.440132    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 14:50:55 +0000 UTC" deadline="2028-04-28 15:45:37.371041723 +0000 UTC"
Nov 02 14:55:56.440373 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.440228    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13032h49m40.930830657s"
Nov 02 14:55:56.453043 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:56.453007    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc1c4566_cd90_43ea_b3cb_6f4b96631b9b.slice/crio-fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43 WatchSource:0}: Error finding container fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43: Status 404 returned error can't find the container with id fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43
Nov 02 14:55:56.453715 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:56.453690    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded89b459_a1e3_4cb7_a05c_1e79ba406239.slice/crio-a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686 WatchSource:0}: Error finding container a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686: Status 404 returned error can't find the container with id a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686
Nov 02 14:55:56.496561 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:56.496527    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffe9e011_abd5_4da2_a109_2eb26c91e182.slice/crio-47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46 WatchSource:0}: Error finding container 47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46: Status 404 returned error can't find the container with id 47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46
Nov 02 14:55:56.496766 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.496748    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:55:56.516159 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:55:56.516122    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba79c206_9b29_4cc1_a3cd_30dc9f2c60e3.slice/crio-08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958 WatchSource:0}: Error finding container 08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958: Status 404 returned error can't find the container with id 08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.950591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:56.950709    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.950839    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.950900    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:58.950881689 +0000 UTC m=+5.275810998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.950965    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:56.951054 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:56.950999    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:58.950988417 +0000 UTC m=+5.275917737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:57.051217 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.051129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:57.051693 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.051365    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:55:57.051693 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.051390    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:55:57.051693 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.051403    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:57.051693 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.051462    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:55:59.051443603 +0000 UTC m=+5.376372916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:57.160649 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.160609    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:57.160827 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.160675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:57.160827 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.160617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:57.160827 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.160769    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:55:57.160997 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.160875    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:55:57.160997 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:57.160945    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:55:57.174291 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.174258    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d59wq" event={"ID":"ffe9e011-abd5-4da2-a109-2eb26c91e182","Type":"ContainerStarted","Data":"47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46"}
Nov 02 14:55:57.177407 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.177370    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xrvk7" event={"ID":"ed89b459-a1e3-4cb7-a05c-1e79ba406239","Type":"ContainerStarted","Data":"a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686"}
Nov 02 14:55:57.179761 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.179725    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerStarted","Data":"fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43"}
Nov 02 14:55:57.185372 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.185331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vsttp" event={"ID":"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3","Type":"ContainerStarted","Data":"08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958"}
Nov 02 14:55:57.440450 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.440377    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 14:50:55 +0000 UTC" deadline="2028-05-03 10:51:17.330103558 +0000 UTC"
Nov 02 14:55:57.440450 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:57.440441    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13147h55m19.889666921s"
Nov 02 14:55:58.969094 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:58.969052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:58.969686 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:58.969115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:58.969686 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:58.969305    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:58.969686 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:58.969372    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:02.969353628 +0000 UTC m=+9.294282968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:55:58.969686 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:58.969443    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:58.969686 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:58.969478    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:02.969465427 +0000 UTC m=+9.294394748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:55:59.070131 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:59.070091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:59.070349 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.070330    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:55:59.070420 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.070355    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:55:59.070420 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.070370    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:59.070527 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.070432    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:03.070412807 +0000 UTC m=+9.395342137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:59.160441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.160567    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:59.160628    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.160710    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:55:59.160751    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:55:59.160876 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:55:59.160813    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:01.160498 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.160217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:01.161009 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:01.160581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:01.161009 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.160685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:01.161009 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:01.160771    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:01.161009 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.160850    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:01.161009 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:01.160923    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:01.214570 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.214535    2580 generic.go:356] "Generic (PLEG): container finished" podID="fd479be79ef05ee5c1afc91f51096ca7" containerID="5c9c874f50dfd5ab684cb69e14a75d48a326bc814f79d9f1ad0c59d54528503d" exitCode=0
Nov 02 14:56:01.214713 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.214632    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerDied","Data":"5c9c874f50dfd5ab684cb69e14a75d48a326bc814f79d9f1ad0c59d54528503d"}
Nov 02 14:56:01.227404 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.227375    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="9dd013b68ca04b66d13824236dde05e3b273552060aa92a4e7711eae91153a94" exitCode=0
Nov 02 14:56:01.227517 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.227461    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"9dd013b68ca04b66d13824236dde05e3b273552060aa92a4e7711eae91153a94"}
Nov 02 14:56:01.236369 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:01.236331    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"6e7f87aa5e9c9e37d740f6ee1d873224","Type":"ContainerStarted","Data":"93860044e1aa061004f22e10c16462cb9ea8ac896532330474e729dbef75e562"}
Nov 02 14:56:02.241443 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.241404    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ldxkl" event={"ID":"bd0c7197-ef21-4d95-94bc-133fbd54c166","Type":"ContainerStarted","Data":"1b071051487467758d566d3692e58cfd9c0696963d8a2c6e5c11a911513d0220"}
Nov 02 14:56:02.245759 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.245726    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vsttp" event={"ID":"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3","Type":"ContainerStarted","Data":"66eb6a4fecd5493a54dba0ca5e50371207ca107c0e6ee2523f9e4aa8433aa785"}
Nov 02 14:56:02.249239 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.249128    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerStarted","Data":"cb68004caf94874b8cf3f091a6abdfad7ac8ae574260844adef7ba1cbd200b7e"}
Nov 02 14:56:02.362930 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.362853    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" podStartSLOduration=8.362832154 podStartE2EDuration="8.362832154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:01.569419836 +0000 UTC m=+7.894349168" watchObservedRunningTime="2026-11-02 14:56:02.362832154 +0000 UTC m=+8.687761485"
Nov 02 14:56:02.478637 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.478564    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ldxkl" podStartSLOduration=2.776591043 podStartE2EDuration="8.478545327s" totalImagesPullingTime="5.701954284s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:02.3640897 +0000 UTC m=+8.689019031" watchObservedRunningTime="2026-11-02 14:56:02.478545327 +0000 UTC m=+8.803474657"
Nov 02 14:56:02.594723 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.593949    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" podStartSLOduration=8.593929772 podStartE2EDuration="8.593929772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:02.479352952 +0000 UTC m=+8.804282282" watchObservedRunningTime="2026-11-02 14:56:02.593929772 +0000 UTC m=+8.918859101"
Nov 02 14:56:02.594723 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:02.594150    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vsttp" podStartSLOduration=3.945151198 podStartE2EDuration="8.594145612s" totalImagesPullingTime="4.648994414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:02.593447041 +0000 UTC m=+8.918376372" watchObservedRunningTime="2026-11-02 14:56:02.594145612 +0000 UTC m=+8.919074942"
Nov 02 14:56:03.006829 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.006787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:03.007027 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.006842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:03.007027 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.006968    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:56:03.007150 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.007031    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:11.007011929 +0000 UTC m=+17.331941244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:56:03.007510 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.007479    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:56:03.007611 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.007533    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:11.007519175 +0000 UTC m=+17.332448486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:56:03.107408 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.107370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:03.107563 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.107501    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:56:03.107563 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.107516    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:56:03.107563 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.107524    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:56:03.107688 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.107573    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:11.107554515 +0000 UTC m=+17.432483823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:56:03.160040 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.160004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:03.160222 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.160136    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:03.160290 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.160235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:03.160349 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.160318    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:03.160406 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.160365    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:03.160618 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:03.160458    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:03.260223 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:03.260132    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74" event={"ID":"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95","Type":"ContainerStarted","Data":"fe275264e368a765ad1455c0d8d374e25d1eac51345a1e12f5c9e4ef65c508d5"}
Nov 02 14:56:04.675216 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:04.673571    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 14:56:05.105178 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.105060    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T14:56:04.673593335Z","UUID":"546ef793-ed14-4121-b863-678238051312","Handler":null,"Name":"","Endpoint":""}
Nov 02 14:56:05.106682 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.106658    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 14:56:05.106818 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.106691    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 14:56:05.160531 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.160495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:05.160531 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.160527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:05.160760 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:05.160649    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:05.160760 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:05.160665    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:05.160760 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.160702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:05.160919 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:05.160783    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:05.267875 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.267838    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74" event={"ID":"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95","Type":"ContainerStarted","Data":"4514e1d2486d926bed16f98aad30126ddbdbf4e28a25ee849ba791a36cb3ae76"}
Nov 02 14:56:05.269839 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.269805    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d59wq" event={"ID":"ffe9e011-abd5-4da2-a109-2eb26c91e182","Type":"ContainerStarted","Data":"26ebe605b66b846a50f7a07f9fc629bf765a6896221a0964dd90689d32426fcc"}
Nov 02 14:56:05.272508 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.272415    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xrvk7" event={"ID":"ed89b459-a1e3-4cb7-a05c-1e79ba406239","Type":"ContainerStarted","Data":"45ca8e10ae621c65ec866192441210428963d7f8514b02ea8bb2f264133cdf41"}
Nov 02 14:56:05.506005 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.505944    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-xrvk7" podStartSLOduration=3.122855721 podStartE2EDuration="11.505928414s" totalImagesPullingTime="8.383072693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:05.505735989 +0000 UTC m=+11.830665321" watchObservedRunningTime="2026-11-02 14:56:05.505928414 +0000 UTC m=+11.830857743"
Nov 02 14:56:05.506315 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.506273    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-d59wq" podStartSLOduration=3.479463904 podStartE2EDuration="11.506259805s" totalImagesPullingTime="8.026795901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:05.38880832 +0000 UTC m=+11.713737650" watchObservedRunningTime="2026-11-02 14:56:05.506259805 +0000 UTC m=+11.831189136"
Nov 02 14:56:05.636503 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.636466    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:05.637372 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:05.637346    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:06.278131 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.277901    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="c4f4a741c1e8beeb8b9b98c678329cddb64cc70d84f92933d2dd01c09627db5b" exitCode=0
Nov 02 14:56:06.278634 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.278099    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"c4f4a741c1e8beeb8b9b98c678329cddb64cc70d84f92933d2dd01c09627db5b"}
Nov 02 14:56:06.280984 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.280957    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74" event={"ID":"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95","Type":"ContainerStarted","Data":"c9c99c40d43ea0e7bea198fb1fe0cd9b90d3d0374b093819e7c2858c2a365a92"}
Nov 02 14:56:06.282531 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.282482    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qs4d5" event={"ID":"84352a64-ea29-427b-a1bb-93bd7aa3298a","Type":"ContainerStarted","Data":"24897532ea5be1ce825241cda61a29feebe6550aa629b2bcec5d397d5108eb47"}
Nov 02 14:56:06.522156 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.521340    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74" podStartSLOduration=2.524896418 podStartE2EDuration="12.521313643s" totalImagesPullingTime="9.996417225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:06.521205896 +0000 UTC m=+12.846135228" watchObservedRunningTime="2026-11-02 14:56:06.521313643 +0000 UTC m=+12.846242973"
Nov 02 14:56:06.635015 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:06.634966    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qs4d5" podStartSLOduration=3.951464312 podStartE2EDuration="12.634949781s" totalImagesPullingTime="8.683485469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:06.634913649 +0000 UTC m=+12.959842980" watchObservedRunningTime="2026-11-02 14:56:06.634949781 +0000 UTC m=+12.959879092"
Nov 02 14:56:07.159880 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:07.159846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:07.159880 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:07.159878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:07.160179 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:07.159965    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:07.160179 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:07.160022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:07.160179 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:07.160122    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:07.160295 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:07.160218    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:07.287038 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:07.286996    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sz7rm" event={"ID":"8cac903e-68cd-473b-9177-e0acbcbee02a","Type":"ContainerStarted","Data":"877c77f636caeee7a5fd53820cbc9b2dc429c78a6c709232090db61d98975f38"}
Nov 02 14:56:07.406676 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:07.406628    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sz7rm" podStartSLOduration=2.523433633 podStartE2EDuration="13.406606257s" totalImagesPullingTime="10.883172624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:07.40610909 +0000 UTC m=+13.731038424" watchObservedRunningTime="2026-11-02 14:56:07.406606257 +0000 UTC m=+13.731535591"
Nov 02 14:56:08.291605 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:08.291568    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="55fca286f05a95f75f51dae49157c2b701bd3e3a1919405bb25669d532bf42d9" exitCode=0
Nov 02 14:56:08.292220 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:08.291660    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"55fca286f05a95f75f51dae49157c2b701bd3e3a1919405bb25669d532bf42d9"}
Nov 02 14:56:09.160407 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:09.160373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:09.160588 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:09.160381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:09.160588 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:09.160485    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:09.160588 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:09.160514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:09.160711 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:09.160606    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:09.160711 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:09.160691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:10.299644 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:10.299445    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="9ca197b4df5e335ef6d80c4e378f49d5d17665a68c1d90b135dda9f5c2776743" exitCode=0
Nov 02 14:56:10.299644 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:10.299538    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"9ca197b4df5e335ef6d80c4e378f49d5d17665a68c1d90b135dda9f5c2776743"}
Nov 02 14:56:11.070383 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.070358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:11.070493 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.070400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:11.070556 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.070494    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:56:11.070556 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.070535    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 14:56:11.070556 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.070548    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:27.070535254 +0000 UTC m=+33.395464580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 14:56:11.070683 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.070587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret podName:fb9bfe09-6645-4ba6-bdbb-62a02364fbb1 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:27.070571727 +0000 UTC m=+33.395501054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret") pod "global-pull-secret-syncer-d7qbm" (UID: "fb9bfe09-6645-4ba6-bdbb-62a02364fbb1") : object "kube-system"/"original-pull-secret" not registered
Nov 02 14:56:11.160389 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.160368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:11.160492 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.160389    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:11.160492 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.160389    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:11.160492 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.160471    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:11.160595 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.160530    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:11.160595 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.160580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:11.170768 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.170747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:11.170938 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.170921    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 14:56:11.171023 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.170944    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 14:56:11.171023 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.170953    2580 projected.go:196] Error preparing data for projected volume kube-api-access-vz46d for pod openshift-network-diagnostics/network-check-target-g5d86: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:56:11.171023 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:11.171000    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d podName:e7beaa44-267d-4887-a21d-077834e70625 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:27.170986569 +0000 UTC m=+33.495915880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vz46d" (UniqueName: "kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d") pod "network-check-target-g5d86" (UID: "e7beaa44-267d-4887-a21d-077834e70625") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 14:56:11.302910 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.302863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:11.303328 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.303229    2580 generic.go:356] "Generic (PLEG): container finished" podID="8e573dd1-b81f-4134-9454-739323385c26" containerID="482381b200dc4d254cf8f3737f12e325b8e1be1b643277ec99731cabb40e84dc" exitCode=1
Nov 02 14:56:11.303328 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.303295    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerDied","Data":"482381b200dc4d254cf8f3737f12e325b8e1be1b643277ec99731cabb40e84dc"}
Nov 02 14:56:11.303405 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:11.303363    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"c23cc4707e1ed3fa8ed578b5f12dffd6e4d4306a3457c1f4ae203fcc4ed7593a"}
Nov 02 14:56:12.309343 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:12.309118    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:12.309833 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:12.309794    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"d37ff8e7f462fa330e78bc5beeeaa6dadacafef63d1e096ec89db47f2e199b2f"}
Nov 02 14:56:12.309888 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:12.309869    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"5708cbbe67573bcf829d654d7502c6a69701b338854356fcdf706acacbd936b7"}
Nov 02 14:56:12.309937 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:12.309908    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"00895afccaa350e1939778441e1cd576fac6296f717899c9c0f16ef63ea01b09"}
Nov 02 14:56:12.310007 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:12.309940    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"05d8cd1ae10175b042a8a42adebcfbeb3901491e7e94c28aa098915e2d5670f3"}
Nov 02 14:56:13.160321 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:13.160278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:13.160481 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:13.160357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:13.160481 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:13.160445    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:13.160481 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:13.160450    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:13.160630 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:13.160536    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:13.160630 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:13.160589    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:13.314880 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:13.314845    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="1fbf7ea81184277c01017f730cd1aa6d3475df3ef190e831797a61be2ba8b9c0" exitCode=0
Nov 02 14:56:13.315471 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:13.314889    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"1fbf7ea81184277c01017f730cd1aa6d3475df3ef190e831797a61be2ba8b9c0"}
Nov 02 14:56:14.320513 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:14.320481    2580 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="2015146dcf90e484ad2d0e2f1e9e3ff0c3f35f4cf6a1c2910fe9be48a839a18e" exitCode=0
Nov 02 14:56:14.321091 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:14.320567    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerDied","Data":"2015146dcf90e484ad2d0e2f1e9e3ff0c3f35f4cf6a1c2910fe9be48a839a18e"}
Nov 02 14:56:14.323975 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:14.323959    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:14.324353 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:14.324332    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"a58c0ae26ab19d3f53b3660e757591725ceb0a4b4e74ca9375d7190f5efab08f"}
Nov 02 14:56:15.159767 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:15.159727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:15.159952 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:15.159727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:15.159952 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:15.159844    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:15.159952 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:15.159745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:15.159952 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:15.159918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:15.160084 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:15.160018    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:15.329462 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:15.329429    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hl98t" event={"ID":"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b","Type":"ContainerStarted","Data":"e53d121af2db2630badd08d46f6751c67ea2d11fba8b29495dd9c81bddff0d18"}
Nov 02 14:56:15.452903 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:15.452859    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hl98t" podStartSLOduration=8.791918041 podStartE2EDuration="21.452846274s" totalImagesPullingTime="12.660928233s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:15.452560397 +0000 UTC m=+21.777489728" watchObservedRunningTime="2026-11-02 14:56:15.452846274 +0000 UTC m=+21.777775601"
Nov 02 14:56:16.334681 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:16.334658    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:16.335347 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:16.335068    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"1532f742cd12f5e17a5382591c32cb76dabf555945f5da62a5ca67a71b36ff43"}
Nov 02 14:56:16.335713 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:16.335693    2580 scope.go:122] "RemoveContainer" containerID="482381b200dc4d254cf8f3737f12e325b8e1be1b643277ec99731cabb40e84dc"
Nov 02 14:56:17.159594 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.159553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:17.159777 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.159568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:17.159777 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.159716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:17.159777 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:17.159708    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:17.163210 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:17.163169    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:17.163365 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:17.163239    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:17.341305 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.341277    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:17.341708 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.341673    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" event={"ID":"8e573dd1-b81f-4134-9454-739323385c26","Type":"ContainerStarted","Data":"503acee26bc607b962ed858d832c887b798d99517c675b916ea8011bd417b231"}
Nov 02 14:56:17.341790 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.341775    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 14:56:17.342106 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.342084    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:17.342203 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.342172    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:17.357471 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.357444    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:17.357812 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.357674    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:17.509263 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.509227    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:17.618413 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:17.618361    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb" podStartSLOduration=8.281608131 podStartE2EDuration="23.61834876s" totalImagesPullingTime="15.336740629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:17.568879816 +0000 UTC m=+23.893809145" watchObservedRunningTime="2026-11-02 14:56:17.61834876 +0000 UTC m=+23.943278090"
Nov 02 14:56:18.488356 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.488321    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d7qbm"]
Nov 02 14:56:18.489083 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.488458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:18.489083 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:18.488563    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:18.491270 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.491241    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g5d86"]
Nov 02 14:56:18.491518 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.491492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:18.491769 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:18.491741    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:18.491902 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.491885    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z6trp"]
Nov 02 14:56:18.491978 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:18.491965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:18.492066 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:18.492051    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:20.160362 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:20.160274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:20.160733 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:20.160274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:20.160733 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:20.160392    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:20.160733 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:20.160404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:20.160733 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:20.160483    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:20.160733 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:20.160543    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:22.127529 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.127434    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:22.127988 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.127562    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 14:56:22.127988 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.127976    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:22.162518 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.162484    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:22.162714 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.162488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:22.162714 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:22.162580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:22.162714 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:22.162662    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:22.162714 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:22.162488    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:22.162920 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:22.162722    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:24.161333 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.161251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:24.161725 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.161355    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:24.161725 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:24.161364    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g5d86" podUID="e7beaa44-267d-4887-a21d-077834e70625"
Nov 02 14:56:24.161725 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:24.161471    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-z6trp" podUID="441ad735-6cb4-45c4-a3a0-cdde58b98a47"
Nov 02 14:56:24.161725 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.161504    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:24.161725 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:24.161574    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d7qbm" podUID="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1"
Nov 02 14:56:24.483016 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.482984    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeReady"
Nov 02 14:56:24.483210 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.483120    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 14:56:24.639059 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.639020    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cqlqj"]
Nov 02 14:56:24.713447 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.713407    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5t56l"]
Nov 02 14:56:24.713614 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.713568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.726039 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.726012    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqlqj"]
Nov 02 14:56:24.726153 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.726063    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5t56l"]
Nov 02 14:56:24.726203 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.726160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.764004 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.763922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.764004 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.763965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9bb9\" (UniqueName: \"kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.766316 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.766298    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 14:56:24.766417 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.766401    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 14:56:24.766880 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.766864    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 14:56:24.767113 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.767102    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wtdgk\""
Nov 02 14:56:24.778265 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.778244    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 14:56:24.778548 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.778532    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 14:56:24.778597 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.778583    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bm8rt\""
Nov 02 14:56:24.864726 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.864726 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.864934 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdlh4\" (UniqueName: \"kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.864934 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.864934 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.865032 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.864969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bb9\" (UniqueName: \"kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.869054 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.869028    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.926536 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.926503    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bb9\" (UniqueName: \"kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:24.966120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.966120 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.966345 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966147    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdlh4\" (UniqueName: \"kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.966345 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.966345 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:24.966263    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:24.966345 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:24.966334    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:25.466316368 +0000 UTC m=+31.791245675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:24.966611 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966583    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:24.966838 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:24.966822    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:25.029670 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:25.029597    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdlh4\" (UniqueName: \"kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:25.034534 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:25.034510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:25.470469 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:25.470429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:25.470875 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:25.470548    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:25.470875 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:25.470600    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:26.470587917 +0000 UTC m=+32.795517238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:25.483633 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:25.483592    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cqlqj"]
Nov 02 14:56:25.487502 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:56:25.487469    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6441f0c0_0287_4d61_bbb6_ac5268f691e3.slice/crio-cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d WatchSource:0}: Error finding container cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d: Status 404 returned error can't find the container with id cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d
Nov 02 14:56:26.160252 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.160210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:26.160458 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.160210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:26.160458 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.160210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:26.213152 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.213115    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 14:56:26.262284 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262242    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 14:56:26.262448 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262376    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dh847\""
Nov 02 14:56:26.262448 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262399    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 14:56:26.262448 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262397    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-ptvxg\""
Nov 02 14:56:26.262599 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262469    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nw765\""
Nov 02 14:56:26.262599 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.262502    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 14:56:26.364503 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.364471    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqlqj" event={"ID":"6441f0c0-0287-4d61-bbb6-ac5268f691e3","Type":"ContainerStarted","Data":"cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d"}
Nov 02 14:56:26.478754 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:26.478712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:26.479234 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:26.478862    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:26.479234 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:26.478926    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:28.478908478 +0000 UTC m=+34.803837800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:27.081691 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.081648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:27.081907 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.081706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:27.081907 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:27.081836    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 14:56:27.082018 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:27.081914    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:59.081893417 +0000 UTC m=+65.406822725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : secret "metrics-daemon-secret" not found
Nov 02 14:56:27.084603 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.084567    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:27.097147 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.097114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:27.182945 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.182902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:27.188216 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.186030    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:27.442787 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.442745    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:27.531790 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.531756    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d7qbm"]
Nov 02 14:56:27.536014 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:56:27.535939    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb9bfe09_6645_4ba6_bdbb_62a02364fbb1.slice/crio-05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94 WatchSource:0}: Error finding container 05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94: Status 404 returned error can't find the container with id 05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94
Nov 02 14:56:27.833459 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:27.833378    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g5d86"]
Nov 02 14:56:27.837469 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:56:27.837420    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7beaa44_267d_4887_a21d_077834e70625.slice/crio-e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb WatchSource:0}: Error finding container e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb: Status 404 returned error can't find the container with id e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb
Nov 02 14:56:28.370776 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:28.370686    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g5d86" event={"ID":"e7beaa44-267d-4887-a21d-077834e70625","Type":"ContainerStarted","Data":"e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb"}
Nov 02 14:56:28.372168 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:28.372131    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d7qbm" event={"ID":"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1","Type":"ContainerStarted","Data":"05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94"}
Nov 02 14:56:28.493598 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:28.493563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:28.493771 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:28.493695    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:28.493771 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:28.493753    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:32.493735581 +0000 UTC m=+38.818664889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:29.378824 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.378785    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cqlqj" event={"ID":"6441f0c0-0287-4d61-bbb6-ac5268f691e3","Type":"ContainerStarted","Data":"4c075f0d82137c1dfb55894fa6aea1b09427373df55b2bd1a19b345eaa13cccd"}
Nov 02 14:56:29.546376 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.546318    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cqlqj" podStartSLOduration=2.212166898 podStartE2EDuration="5.546304375s" totalImagesPullingTime="3.334137477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:29.496931597 +0000 UTC m=+35.821860931" watchObservedRunningTime="2026-11-02 14:56:29.546304375 +0000 UTC m=+35.871233702"
Nov 02 14:56:29.546533 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.546473    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fjzx9"]
Nov 02 14:56:29.573464 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.573434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.600637 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.600606    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjzx9"]
Nov 02 14:56:29.645868 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.645772    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 14:56:29.646100 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.646071    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fbnz4\""
Nov 02 14:56:29.706111 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706073    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 14:56:29.706300 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.706300 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.706300 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.706300 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706293    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 14:56:29.706451 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706305    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z84s7\" (UniqueName: \"kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.706451 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706297    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 14:56:29.706451 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.706405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.807070 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.807281 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.807281 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.807281 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.807281 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z84s7\" (UniqueName: \"kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:29.807564    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:29.807622    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:30.307609506 +0000 UTC m=+36.632538815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.807920    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:29.807948    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:29.808002    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:30.30798862 +0000 UTC m=+36.632917950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:29.808319 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.808275    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.897699 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.897625    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z84s7\" (UniqueName: \"kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:29.936625 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.936267    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"]
Nov 02 14:56:29.951616 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:29.950882    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:30.002462 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.002376    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"]
Nov 02 14:56:30.005314 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.005284    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 14:56:30.005477 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.005384    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 14:56:30.005575 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.005501    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mcvdb\""
Nov 02 14:56:30.008919 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.008890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n249p\" (UniqueName: \"kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p\") pod \"migrator-66df5d496b-4lshl\" (UID: \"8d5ec2fd-9039-46e6-924e-9f60195c478b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:30.110093 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.110059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n249p\" (UniqueName: \"kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p\") pod \"migrator-66df5d496b-4lshl\" (UID: \"8d5ec2fd-9039-46e6-924e-9f60195c478b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:30.173748 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.173679    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n249p\" (UniqueName: \"kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p\") pod \"migrator-66df5d496b-4lshl\" (UID: \"8d5ec2fd-9039-46e6-924e-9f60195c478b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:30.279657 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.279616    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:30.312751 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.312722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:30.312873 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.312806    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:30.312945 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:30.312911    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:30.313001 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:30.312971    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:31.312956705 +0000 UTC m=+37.637886013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:30.313237 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:30.313205    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:30.313368 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:30.313286    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:31.313266411 +0000 UTC m=+37.638195740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:30.683181 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:30.683138    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"]
Nov 02 14:56:30.687584 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:56:30.687543    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5ec2fd_9039_46e6_924e_9f60195c478b.slice/crio-c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249 WatchSource:0}: Error finding container c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249: Status 404 returned error can't find the container with id c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249
Nov 02 14:56:31.152614 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.151682    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-g8pcw"]
Nov 02 14:56:31.172817 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.172782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.215205 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.215105    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-g8pcw"]
Nov 02 14:56:31.225443 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.225411    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 14:56:31.225618 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.225601    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zslfv\""
Nov 02 14:56:31.295719 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.295686    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 14:56:31.295898 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.295836    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 14:56:31.295898 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.295887    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 14:56:31.320958 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.320911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:31.321152 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.321009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:31.321152 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.321055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.321152 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.321088    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2jv4\" (UniqueName: \"kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.321152 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.321115    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.321369 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:31.321058    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:31.321369 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:31.321145    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:31.321369 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:31.321246    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:33.321225647 +0000 UTC m=+39.646154956 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:31.321369 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:31.321264    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:33.321256829 +0000 UTC m=+39.646186142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:31.386379 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.386340    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl" event={"ID":"8d5ec2fd-9039-46e6-924e-9f60195c478b","Type":"ContainerStarted","Data":"c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249"}
Nov 02 14:56:31.422075 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.421788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.422075 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.421837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2jv4\" (UniqueName: \"kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.422075 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.421865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.423051 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.422995    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.424847 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.424823    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.483845 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.483797    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2jv4\" (UniqueName: \"kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.498715 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.498468    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:31.913591 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:31.913503    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-g8pcw"]
Nov 02 14:56:31.972615 ip-10-0-139-197 kubenswrapper[2580]: W1102 14:56:31.972581    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf641dd2a_3e2d_474c_a777_758342eebfb0.slice/crio-e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17 WatchSource:0}: Error finding container e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17: Status 404 returned error can't find the container with id e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17
Nov 02 14:56:32.390819 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:32.390773    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw" event={"ID":"f641dd2a-3e2d-474c-a777-758342eebfb0","Type":"ContainerStarted","Data":"e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17"}
Nov 02 14:56:32.392676 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:32.392643    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g5d86" event={"ID":"e7beaa44-267d-4887-a21d-077834e70625","Type":"ContainerStarted","Data":"ea8aebf5c0109b6b753ebbd063230c9cb1c9a8e28af59a375801ce5279d369e7"}
Nov 02 14:56:32.392814 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:32.392797    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:32.513214 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:32.513081    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g5d86" podStartSLOduration=34.338810144 podStartE2EDuration="38.513059641s" totalImagesPullingTime="4.174249497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:55:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:32.512515895 +0000 UTC m=+38.837445225" watchObservedRunningTime="2026-11-02 14:56:32.513059641 +0000 UTC m=+38.837988975"
Nov 02 14:56:32.530034 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:32.530000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:32.530291 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:32.530135    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:32.530291 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:32.530215    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:40.530182802 +0000 UTC m=+46.855112127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:33.336792 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:33.336755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:33.337354 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:33.336816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:33.337354 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:33.336908    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:33.337354 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:33.336961    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:37.336948784 +0000 UTC m=+43.661878096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:33.337354 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:33.336997    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:33.337354 ip-10-0-139-197 kubenswrapper[2580]: E1102 14:56:33.337026    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:37.337017469 +0000 UTC m=+43.661946780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:33.397881 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:33.397786    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d7qbm" event={"ID":"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1","Type":"ContainerStarted","Data":"ea1e1847dc87829e4f79fbea767be29cb35d20cd63580a97b441b7f20ffb26c0"}
Nov 02 14:56:33.399617 ip-10-0-139-197 kubenswrapper[2580]: I1102 14:56:33.399577    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl" event={"ID":"8d5ec2fd-9039-46e6-924e-9f60195c478b","Type":"ContainerStarted","Data":"6bcbbef7a6329e758a7f8aeb12b58184e47635bee502216dc073249bc1ec1b72"}
Nov 02 14:56:33.504167 ip-10-0-139-197 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 14:56:33.515898 ip-10-0-139-197 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 14:56:33.516140 ip-10-0-139-197 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 14:56:33.516228 ip-10-0-139-197 systemd[1]: kubelet.service: Consumed 2.409s CPU time.
Nov 02 14:56:33.531752 ip-10-0-139-197 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 14:56:33.616735 ip-10-0-139-197 kubenswrapper[4470]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:56:33.616735 ip-10-0-139-197 kubenswrapper[4470]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 14:56:33.616735 ip-10-0-139-197 kubenswrapper[4470]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:56:33.616735 ip-10-0-139-197 kubenswrapper[4470]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 14:56:33.618979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618893    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.618979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618979    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.618979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618983    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.618979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618986    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618989    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618992    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618995    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.618998    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619001    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619004    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619007    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619009    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619012    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619015    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619017    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619020    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619022    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619025    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619028    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619030    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619033    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619036    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.619103 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619038    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619041    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619043    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619046    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619056    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619059    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619063    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619065    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619068    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619071    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619074    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619077    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619083    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619086    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619089    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619093    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619097    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619101    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619104    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.619620 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619108    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619111    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619114    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619117    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619120    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619122    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619125    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619128    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619131    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619134    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619137    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619140    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619142    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619146    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619149    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619152    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619160    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619163    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619166    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619168    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.620108 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619171    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619173    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619176    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619179    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619182    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619199    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619202    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619204    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619207    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619210    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619213    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619215    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619218    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619220    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619223    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619226    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619229    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619231    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619234    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619237    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.620624 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619240    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619242    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619245    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619248    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619251    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619254    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619257    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619260    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619263    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619266    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619269    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619271    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619274    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619276    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619279    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619282    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619284    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619287    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619290    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.621136 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619297    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619302    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619305    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619308    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619311    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.619314    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620128    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620135    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620138    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620142    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620144    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620148    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620151    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620154    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620157    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620159    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620162    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620166    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620168    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620171    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.621679 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620174    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620176    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620179    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620183    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620201    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620204    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620207    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620210    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620212    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620215    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620218    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620220    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620223    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620226    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620229    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620232    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620235    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620237    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620240    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620242    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.622182 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620245    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620248    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620251    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620254    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620257    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620259    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620262    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620265    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620267    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620270    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620273    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620275    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620278    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620281    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620283    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620286    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620289    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620291    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620294    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620296    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.622718 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620299    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620301    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620304    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620306    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620309    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620311    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620316    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620319    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620322    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620324    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620327    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620330    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620333    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620336    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620338    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620341    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620344    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620347    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620349    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620352    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.623263 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620356    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620360    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620364    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620366    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620371    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620375    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620378    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620381    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620383    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620386    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620389    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620391    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620394    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620396    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620399    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620402    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620405    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620407    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620412    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.623773 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620415    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620418    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620421    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620423    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620426    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620428    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620431    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620433    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620436    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620439    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620441    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620444    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620446    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620973    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620979    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620983    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620985    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620989    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620991    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.624264 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620994    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.620997    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621000    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621003    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621006    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621009    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621013    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621015    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621019    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621022    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621024    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621027    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621030    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621033    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621036    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621038    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621040    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621043    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621045    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.624748 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621048    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621050    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621053    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621056    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621058    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621061    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621064    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621066    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621070    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621074    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621078    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621080    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621083    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621085    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621088    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621091    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621094    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621097    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621100    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.625276 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621102    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621105    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621108    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621110    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621113    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621115    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621118    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621121    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621123    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621126    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621128    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621131    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621134    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621136    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621139    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621142    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621144    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621148    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621152    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.625882 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621155    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621158    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621161    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621164    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621166    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621169    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621171    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621174    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621176    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621179    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621181    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621184    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621201    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621204    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621206    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621209    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621212    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621214    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621217    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621220    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.626733 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621222    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621225    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621228    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621230    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621233    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621236    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621238    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621241    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621243    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621246    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621248    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621251    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621255    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621257    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621260    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621262    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621265    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621267    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621270    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621272    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.627490 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621275    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621278    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.621281    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621378    4470 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621386    4470 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621394    4470 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621399    4470 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621405    4470 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621409    4470 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621414    4470 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621418    4470 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621422    4470 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621426    4470 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621432    4470 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621436    4470 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621440    4470 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621444    4470 flags.go:64] FLAG: --cgroup-root=""
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621447    4470 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621450    4470 flags.go:64] FLAG: --client-ca-file=""
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621454    4470 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621457    4470 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621461    4470 flags.go:64] FLAG: --cluster-domain=""
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621464    4470 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 14:56:33.628044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621467    4470 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621471    4470 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621474    4470 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621479    4470 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621482    4470 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621486    4470 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621489    4470 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621492    4470 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621496    4470 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621499    4470 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621503    4470 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621506    4470 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621510    4470 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621513    4470 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621518    4470 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621522    4470 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621525    4470 flags.go:64] FLAG: --enable-server="true"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621529    4470 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621534    4470 flags.go:64] FLAG: --event-burst="100"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621538    4470 flags.go:64] FLAG: --event-qps="50"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621541    4470 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621544    4470 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621549    4470 flags.go:64] FLAG: --eviction-hard=""
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621555    4470 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 14:56:33.628623 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621558    4470 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621562    4470 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621565    4470 flags.go:64] FLAG: --eviction-soft=""
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621569    4470 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621572    4470 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621575    4470 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621578    4470 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621581    4470 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621584    4470 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621587    4470 flags.go:64] FLAG: --feature-gates=""
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621591    4470 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621594    4470 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621598    4470 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621601    4470 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621604    4470 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621607    4470 flags.go:64] FLAG: --help="false"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621610    4470 flags.go:64] FLAG: --hostname-override="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621613    4470 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621616    4470 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621619    4470 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621623    4470 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621626    4470 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621631    4470 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 14:56:33.629243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621634    4470 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621638    4470 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621641    4470 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621644    4470 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621647    4470 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621650    4470 flags.go:64] FLAG: --kube-reserved=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621653    4470 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621658    4470 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621661    4470 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621664    4470 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621667    4470 flags.go:64] FLAG: --lock-file=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621670    4470 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621673    4470 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621676    4470 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621683    4470 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621686    4470 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621689    4470 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621692    4470 flags.go:64] FLAG: --logging-format="text"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621695    4470 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621699    4470 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621702    4470 flags.go:64] FLAG: --manifest-url=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621705    4470 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621710    4470 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621713    4470 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621717    4470 flags.go:64] FLAG: --max-pods="110"
Nov 02 14:56:33.629941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621720    4470 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621723    4470 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621726    4470 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621728    4470 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621731    4470 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621735    4470 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621739    4470 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621747    4470 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621750    4470 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621753    4470 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621757    4470 flags.go:64] FLAG: --pod-cidr=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621760    4470 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621763    4470 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621767    4470 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621771    4470 flags.go:64] FLAG: --port="10250"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621774    4470 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621777    4470 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-03d898ffef658186f"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621781    4470 flags.go:64] FLAG: --qos-reserved=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621784    4470 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621787    4470 flags.go:64] FLAG: --register-node="true"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621790    4470 flags.go:64] FLAG: --register-with-taints=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621794    4470 flags.go:64] FLAG: --registry-burst="10"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621797    4470 flags.go:64] FLAG: --registry-qps="5"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621800    4470 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621803    4470 flags.go:64] FLAG: --reserved-memory=""
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621807    4470 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 14:56:33.630690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621810    4470 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621813    4470 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621816    4470 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621819    4470 flags.go:64] FLAG: --runonce="false"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621822    4470 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621825    4470 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621828    4470 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621831    4470 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621834    4470 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621837    4470 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621840    4470 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621844    4470 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621848    4470 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621851    4470 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621854    4470 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621857    4470 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621861    4470 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621864    4470 flags.go:64] FLAG: --system-cgroups=""
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621867    4470 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621873    4470 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621877    4470 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621880    4470 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621884    4470 flags.go:64] FLAG: --tls-min-version=""
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621887    4470 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621890    4470 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 14:56:33.631558 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621893    4470 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621896    4470 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621900    4470 flags.go:64] FLAG: --v="2"
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621904    4470 flags.go:64] FLAG: --version="false"
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621909    4470 flags.go:64] FLAG: --vmodule=""
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621913    4470 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.621917    4470 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622070    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622075    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622078    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622080    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622083    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622086    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622088    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622091    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622093    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622097    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622099    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622102    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622108    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622111    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.632214 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622113    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622116    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622119    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622122    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622124    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622127    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622130    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622133    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622136    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622139    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622141    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622144    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622146    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622149    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622152    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622155    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622157    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622160    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622162    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.632746 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622165    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622168    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622170    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622173    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622175    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622178    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622180    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622183    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622198    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622202    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622206    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622212    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622216    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622218    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622221    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622224    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622226    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622230    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622232    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.633291 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622237    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622240    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622243    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622245    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622248    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622251    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622253    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622257    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622260    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622264    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622266    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622269    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622272    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622274    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622277    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622279    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622282    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622285    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622287    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.633896 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622290    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622293    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622295    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622298    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622301    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622305    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622308    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622310    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622313    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622315    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622318    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622321    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622324    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622328    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622330    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622333    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622335    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622338    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622341    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622343    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.634550 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622346    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622349    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622352    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622354    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622356    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622359    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622361    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622364    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622367    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622369    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622372    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622374    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622377    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622380    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622382    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.622410    4470 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:56:33.635138 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622445    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622449    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622454    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622457    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622460    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622463    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622466    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622468    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622471    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622475    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622478    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622481    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622484    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622486    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622489    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622492    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622494    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622497    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622499    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622502    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.635617 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622505    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622507    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622510    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622512    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622515    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622517    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622520    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622522    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622525    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622528    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622530    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622533    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622536    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622538    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622542    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622545    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622548    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622550    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622553    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622555    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.636320 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622558    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622562    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622565    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622568    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622571    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622573    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622576    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622578    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622581    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622584    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622587    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622589    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622592    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622601    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622605    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622608    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622611    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622613    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622616    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.636941 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622618    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622621    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622624    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622627    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622630    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622632    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622635    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622639    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622642    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622644    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622647    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622649    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622652    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622655    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622658    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622661    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622665    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622667    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622670    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.637474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622675    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622678    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622682    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622685    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622688    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622690    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622693    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622695    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622698    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622701    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622703    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622706    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622708    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622711    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622713    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622716    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622720    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622725    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622728    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.637945 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622731    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622735    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622738    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622742    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622744    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622747    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622751    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622753    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.622757    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.628345    4470 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.628366    4470 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.628393    4470 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.628398    4470 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628473    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628479    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628483    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628486    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628489    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628492    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.638448 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628495    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628498    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628501    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628504    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628507    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628510    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628513    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628516    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628518    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628522    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628526    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628528    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628531    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628534    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628536    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628539    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628542    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628544    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628547    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.638935 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628550    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628554    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628556    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628560    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628563    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628565    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628568    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628571    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628574    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628576    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628579    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628582    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628585    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628588    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628592    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628596    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628599    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628603    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628606    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628609    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.639426 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628612    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628616    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628619    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628622    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628625    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628628    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628631    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628635    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628638    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628641    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628643    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628646    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628649    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628652    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628654    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628657    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628660    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628662    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628665    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628668    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.639964 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628670    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628673    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628676    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628678    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628681    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628684    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628686    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628689    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628691    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628694    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628697    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628699    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628701    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628704    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628707    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628710    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628713    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628716    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628719    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628722    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.640482 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628724    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628727    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628730    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628732    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628735    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628738    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628740    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628743    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628747    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628751    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628753    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628756    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628759    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628761    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628764    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628766    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628769    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628772    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628774    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.640979 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628777    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628780    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.628809    4470 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628957    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628962    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628965    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628969    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628972    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628976    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628979    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628981    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628984    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628987    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628991    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628996    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.641560 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.628999    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629002    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629005    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629008    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629011    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629014    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629016    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629019    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629022    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629025    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629028    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629030    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629033    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629035    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629038    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629041    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629043    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629046    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629049    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629051    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.641938 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629054    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629057    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629059    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629062    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629065    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629067    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629070    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629073    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629076    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629079    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629083    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629085    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629088    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629091    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629093    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629096    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629098    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629101    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629104    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629106    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.642445 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629109    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629111    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629114    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629116    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629119    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629121    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629124    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629127    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629129    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629132    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629134    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629137    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629141    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629144    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629147    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629150    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629153    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629156    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629158    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629162    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.642933 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629164    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629167    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629170    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629172    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629175    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629178    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629180    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629182    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629204    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629207    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629210    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629213    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629216    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629219    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629221    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629224    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629227    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629230    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629233    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629236    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.643431 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629239    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629241    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629244    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629247    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629250    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629253    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629256    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629259    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629261    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629264    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629267    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629269    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629273    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629275    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.629301    4470 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629336    4470 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 14:56:33.643910 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629341    4470 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629345    4470 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629348    4470 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629350    4470 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629353    4470 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629356    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629358    4470 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629361    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629364    4470 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629366    4470 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629369    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629371    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629374    4470 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629376    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629379    4470 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629382    4470 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629385    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629387    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629391    4470 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 14:56:33.644350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629393    4470 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629396    4470 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629399    4470 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629401    4470 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629404    4470 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629406    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629409    4470 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629412    4470 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629416    4470 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629419    4470 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629422    4470 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629424    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629427    4470 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629430    4470 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629432    4470 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629435    4470 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629437    4470 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629440    4470 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629442    4470 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 14:56:33.645046 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629445    4470 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629447    4470 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629450    4470 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629453    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629455    4470 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629458    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629461    4470 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629463    4470 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629466    4470 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629469    4470 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629471    4470 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629474    4470 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629487    4470 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629490    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629492    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629495    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629498    4470 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629500    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629503    4470 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629506    4470 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 14:56:33.645827 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629508    4470 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629511    4470 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629513    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629516    4470 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629520    4470 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629523    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629525    4470 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629528    4470 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629530    4470 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629533    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629535    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629538    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629545    4470 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629547    4470 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629550    4470 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629552    4470 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629562    4470 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629565    4470 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629568    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629570    4470 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 14:56:33.646382 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629573    4470 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629576    4470 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629580    4470 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629584    4470 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629587    4470 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629590    4470 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629593    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629595    4470 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629598    4470 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629600    4470 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629603    4470 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629606    4470 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629608    4470 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629611    4470 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629613    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629616    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629619    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629622    4470 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629625    4470 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 14:56:33.646885 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629627    4470 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629630    4470 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629632    4470 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629635    4470 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629638    4470 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629642    4470 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629646    4470 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:33.629649    4470 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.629714    4470 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.630766    4470 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.630829    4470 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.631007    4470 server.go:1014] "Starting client certificate rotation"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.631148    4470 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.631212    4470 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 14:50:54 +0000 UTC" deadline="2028-08-02 09:49:39.74360525 +0000 UTC"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.631251    4470 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15330h53m6.112356275s"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.631480    4470 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 14:56:33.647363 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.632607    4470 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 14:56:33.647840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.634162    4470 log.go:25] "Validated CRI v1 runtime API"
Nov 02 14:56:33.647840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.635886    4470 log.go:25] "Validated CRI v1 image API"
Nov 02 14:56:33.647840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.636677    4470 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 14:56:33.647840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.638358    4470 fs.go:135] Filesystem UUIDs: map[2cfa3683-7091-4562-81cc-36493a3ce05b:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c7eb937a-6db3-4ff7-ae22-4aaef4a89837:/dev/nvme0n1p3]
Nov 02 14:56:33.648000 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.638379    4470 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/072126f8-4f44-46b1-a1ca-221a57d7feb3/volumes/kubernetes.io~projected/kube-api-access-tdlh4:{mountpoint:/var/lib/kubelet/pods/072126f8-4f44-46b1-a1ca-221a57d7feb3/volumes/kubernetes.io~projected/kube-api-access-tdlh4 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/441ad735-6cb4-45c4-a3a0-cdde58b98a47/volumes/kubernetes.io~projected/kube-api-access-ts97h:{mountpoint:/var/lib/kubelet/pods/441ad735-6cb4-45c4-a3a0-cdde58b98a47/volumes/kubernetes.io~projected/kube-api-access-ts97h major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~projected/kube-api-access-r9bb9:{mountpoint:/var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~projected/kube-api-access-r9bb9 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d80c35b-aefd-4e05-b331-f30b815e11e6/volumes/kubernetes.io~projected/kube-api-access-z84s7:{mountpoint:/var/lib/kubelet/pods/7d80c35b-aefd-4e05-b331-f30b815e11e6/volumes/kubernetes.io~projected/kube-api-access-z84s7 major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/84352a64-ea29-427b-a1bb-93bd7aa3298a/volumes/kubernetes.io~projected/kube-api-access-ztmd9:{mountpoint:/var/lib/kubelet/pods/84352a64-ea29-427b-a1bb-93bd7aa3298a/volumes/kubernetes.io~projected/kube-api-access-ztmd9 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~projected/kube-api-access-9x7t6:{mountpoint:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~projected/kube-api-access-9x7t6 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d5ec2fd-9039-46e6-924e-9f60195c478b/volumes/kubernetes.io~projected/kube-api-access-n249p:{mountpoint:/var/lib/kubelet/pods/8d5ec2fd-9039-46e6-924e-9f60195c478b/volumes/kubernetes.io~projected/kube-api-access-n249p major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~projected/kube-api-access-hvdz5:{mountpoint:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~projected/kube-api-access-hvdz5 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95/volumes/kubernetes.io~projected/kube-api-access-8zpb8:{mountpoint:/var/lib/kubelet/pods/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95/volumes/kubernetes.io~projected/kube-api-access-8zpb8 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3/volumes/kubernetes.io~secret/agent-certs major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b/volumes/kubernetes.io~projected/kube-api-access-5p8z7:{mountpoint:/var/lib/kubelet/pods/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b/volumes/kubernetes.io~projected/kube-api-access-5p8z7 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bd0c7197-ef21-4d95-94bc-133fbd54c166/volumes/kubernetes.io~projected/kube-api-access-sd7ww:{mountpoint:/var/lib/kubelet/pods/bd0c7197-ef21-4d95-94bc-133fbd54c166/volumes/kubernetes.io~projected/kube-api-access-sd7ww major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e7beaa44-267d-4887-a21d-077834e70625/volumes/kubernetes.io~projected/kube-api-access-vz46d:{mountpoint:/var/lib/kubelet/pods/e7beaa44-267d-4887-a21d-077834e70625/volumes/kubernetes.io~projected/kube-api-access-vz46d major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed89b459-a1e3-4cb7-a05c-1e79ba406239/volumes/kubernetes.io~projected/kube-api-access-tvz6c:{mountpoint:/var/lib/kubelet/pods/ed89b459-a1e3-4cb7-a05c-1e79ba406239/volumes/kubernetes.io~projected/kube-api-access-tvz6c major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~projected/kube-api-access-v2jv4:{mountpoint:/var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~projected/kube-api-access-v2jv4 major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~secret/signing-key major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ffe9e011-abd5-4da2-a109-2eb26c91e182/volumes/kubernetes.io~projected/kube-api-access-bxfqw:{mountpoint:/var/lib/kubelet/pods/ffe9e011-abd5-4da2-a109-2eb26c91e182/volumes/kubernetes.io~projected/kube-api-access-bxfqw major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d1520cbeee2e132c08c076444c167b47820ada3e88adb75a29bd912643d2b206/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/0d72d66027f331293a5bbc381dce80c9ea04ee5aee805502c60a2baa77e6929f/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/ea8d14204e1321a35e176dc7a6cf60fa42df3261edc8cab680d28146159a09df/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/45900d70d4239bf066a87a2d7377440c79b42ef7195163abae082c3f5f0f2473/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/2e7ff697c707c9f280ecb53084be50b6048ceaa3424ba6c355bef356429e97b9/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-118:{mountpoint:/var/lib/containers/storage/overlay/41b80ba948d601ee9e8cbea57f7259d67d9fb483d553c125f04fdc1574b4f619/merged major:0 minor:118 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/7b3a0ac458f956f06d868ea069e841ec2e36ff3d62330509cf9ed5f979f024cc/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/4f50c6f4461e342258d2381f1e75ce75ff7388ae19f66dbc2422ea1f322ffa0c/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/24d5a67b5d5a00c7d8f215096732a47e30e630d6380b1fa063c0e25b175fc162/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/894af10b2dfaac826cbb7aae360d63d5ea4c83b7821f2c3e52b1b4563f0971a0/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/8d0ebc0ee5825a136548095105f4088f5f02bb7f0b4ea087280f90e58f290ceb/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/19ef37232a59e98cfea459e19e0a62dde8813c5269b2ed5f686fe607505333b1/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/9be6c042bb6630e98394350f276d5d6aa3bd0a8a8f88ff143bb0d81cf0dd0c56/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/08be623085dfb6d3ac7f5f340f3bcca63c8493afb1761977062774aa17d9929e/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/16e5190b9f324fba84edef14958e8671cfecfa1ca7fe88dd12f78a007d0c2b1a/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/99a6355cf1905a9b6390c000039d2b8945470154c83c094c33ff5f806091addd/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/40f552397066ab69b05095c0c39ff7f507fb21a30db7ca122594e098a6398959/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/df4f1f117404a56e8b895a20352415e54ddd7991fe86bfb3267ec7858268d807/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/008e591bce21ad252c88eabc053c77d430415aa5dabaf1455095470ca2017efd/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/9ddca9918b6f891db29358bd0690d463986d8d58bc82922e27991096a8076c15/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/c7067c1e6350c9d4aa7ef93ea493b2329f26b9c4c51babb57092aecae2a280b1/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/a2908bd6cda836a4b56141c0f708e06c26b8353232477d1a8ab8c9b35b6011cb/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/f79073ef5b26825478fb645e62157c6f4b60d05dbedbc4425cc4c9fd90656a94/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/0efa54cd15935a83b8eba38610cb762d90f9a626960bc82c959cbeef77060fb6/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-216:{mountpoint:/var/lib/containers/storage/overlay/c5738ce814992b4d75d5e6eee989e09d1b536af98c96313666468e2f833774a3/merged major:0 minor:216 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/7166fe91dbeb83f832a13d68ceb9396290594068b94de6ed535fa21cb1e86e75/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/3785f64db8f3c84b80a34a7a071fb1658b54c678706c16e7497120911aed1981/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-234:{mountpoint:/var/lib/containers/storage/overlay/ffbda329ce9a9132a58a3367fb84168f516823d6674f36b70a953755e167b55f/merged major:0 minor:234 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/f3a63d0313eaaabb7b03126cfcec019c4ac522ffb3029027014a4cfc529bdd5e/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/6d1f715720f30db6c6626834be7e112ade429b8b8c988e818d2511a0dfb9c012/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/b882d36e5ddf1032dd33a9737822567bf7adbbd0df530ac46a687e8a8351d64a/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/b4c04a0d2ecdf304156930e12c972155269dcd69a55a52c725706002e0beaecb/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/b4070109523436796795f6f3cb3481f29701eb94d88ce671e8100f5f141af567/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/5b1775028fdc74854858e5491e8b05be725ff3ac1c43d35877df5f9b45341173/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/c51ae0e370d75a3d9060475c12ae5ac5ca10e75267a02a6e043e2a7f77779609/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/4c5a2303b7dd2a9ed2d6b72f86518e416e0b5ecd8fbfbb3b6ebea202697e230c/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/279be693039124eab91b9543e66c33f4ccea545f619494fb107c81f84e1566d2/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/7a6f0601b727ab2ce0257dff4c7361cb6af4ccb6dc00fa288006f0fca8f987d8/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/840fa86611e19f242c867e0035aa4c23aecec5ebc77494a2016950088068b807/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/72fb903988c774ba88917568e075e80e1e92ef5c43a72beb72a9594c5a7de9c4/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.645768    4470 manager.go:217] Machine: {Timestamp:2026-11-02 14:56:33.644956628 +0000 UTC m=+0.082858762 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099991 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b4cee81dff1443501dfc355863f1d SystemUUID:ec2b4cee-81df-f144-3501-dfc355863f1d BootID:87227f92-df8e-4e91-9175-f17f34b5e512 Filesystems:[{Device:/run/containers/storage/overlay-containers/e28b59f4fac56749acfea26d5b21db9131ac8d856a1c7ade4701d220be0408eb/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-234 DeviceMajor:0 DeviceMinor:234 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd3ec3ba3e516c5e2ae4bd5f328e0a0f305d7c503ebed6f79ddfb5f7185d622d/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a086268099cad5ec7c8bfe1237c049c4c02f7478800ae8f512471099fcf34bc/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6441f0c0-0287-4d61-bbb6-ac5268f691e3/volumes/kubernetes.io~projected/kube-api-access-r9bb9 DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-216 DeviceMajor:0 DeviceMinor:216 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95/volumes/kubernetes.io~projected/kube-api-access-8zpb8 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/05005a3698308c3790dc392b28fa62feae501c919c81c22140adad30d2f19d94/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e7beaa44-267d-4887-a21d-077834e70625/volumes/kubernetes.io~projected/kube-api-access-vz46d DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8d5ec2fd-9039-46e6-924e-9f60195c478b/volumes/kubernetes.io~projected/kube-api-access-n249p DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ffe9e011-abd5-4da2-a109-2eb26c91e182/volumes/kubernetes.io~projected/kube-api-access-bxfqw DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d80c35b-aefd-4e05-b331-f30b815e11e6/volumes/kubernetes.io~projected/kube-api-access-z84s7 DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volumes/kubernetes.io~projected/kube-api-access-hvdz5 DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b631f8c7fd3264bf680ef591659b08b7843d01339f8ac20c286c24e1696ecff1/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee62acb481faa74d4f5fe665eadb5dcc38438a0765783e0ebe4a76d918266f46/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/47d804b5b87f0ff5b2fe318ea78aed25fa37471fbc739db6bd1da172ece86a46/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e71af71cdc5b49dae86fed95c19eb21e83028b593e2a20d1fa7b938140c18b17/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-118 DeviceMajor:0 DeviceMinor:118 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~projected/kube-api-access-9x7t6 DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/84352a64-ea29-427b-a1bb-93bd7aa3298a/volumes/kubernetes.io~projected/kube-api-access-ztmd9 DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b/volumes/kubernetes.io~projected/kube-api-access-5p8z7 DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8e573dd1-b81f-4134-9454-739323385c26/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a8cab69bf0bdea7c8dbcfcebb4517eb6b279e633b16bbd3c2e6a1a9d0e1a2686/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c7b0a15d6bfbf5d74d27331a858b88a956351650425fefd693dafc0a8f3a5249/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/441ad735-6cb4-45c4-a3a0-cdde58b98a47/volumes/kubernetes.io~projected/kube-api-access-ts97h DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/072126f8-4f44-46b1-a1ca-221a57d7feb3/volumes/kubernetes.io~projected/kube-api-access-tdlh4 DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/22d126024910cb812fddb422f190bb53d1511885dac4f4d3072d1dcba51679f5/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/08fe94059a1da03ab1e4e22f69b608bd376af4d1d479ae7aa4eb7d8594192958/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/bd0c7197-ef21-4d95-94bc-133fbd54c166/volumes/kubernetes.io~projected/kube-api-access-sd7ww DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ed89b459-a1e3-4cb7-a05c-1e79ba406239/volumes/kubernetes.io~projected/kube-api-access-tvz6c DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f641dd2a-3e2d-474c-a777-758342eebfb0/volumes/kubernetes.io~projected/kube-api-access-v2jv4 DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8cac903e-68cd-473b-9177-e0acbcbee02a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb45807e0ad883f2ce81a044c130afeeea09dfddb40a8696e86864fe06acff43/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:05005a3698308c3 MacAddress:66:9e:2a:fa:fd:d2 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:13:10:c9:82:29 Speed:0 Mtu:9001} {Name:br-int MacAddress:36:e0:db:58:79:59 Speed:0 Mtu:8901} {Name:c7b0a15d6bfbf5d MacAddress:f2:67:ad:d3:03:b4 Speed:10000 Mtu:8901} {Name:cd3ec3ba3e516c5 MacAddress:0e:56:dd:96:e7:e9 Speed:10000 Mtu:8901} {Name:e28b59f4fac5674 MacAddress:d6:24:7c:19:b5:11 Speed:10000 Mtu:8901} {Name:e71af71cdc5b49d MacAddress:e6:53:64:be:f8:51 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:13:10:c9:82:29 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:6a:57:02:6f:31:65 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:0a:7b:d5:3f:dc:b5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.646410    4470 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.646481    4470 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.646820    4470 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.646847    4470 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-197.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647003    4470 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647010    4470 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647021    4470 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647037    4470 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647145    4470 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647336    4470 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647448    4470 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647485    4470 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647495    4470 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647505    4470 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647514    4470 kubelet.go:405] "Adding apiserver pod source"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.647523    4470 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.648088    4470 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 14:56:33.648159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.648098    4470 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 14:56:33.648840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.648607    4470 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 14:56:33.657697 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.657674    4470 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 14:56:33.657697 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.657694    4470 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 14:56:33.657889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.657838    4470 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 14:56:33.658100 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658083    4470 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 14:56:33.658179 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658124    4470 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 14:56:33.658281 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658269    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658291    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658301    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658310    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658321    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658331    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 14:56:33.658346 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658341    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658350    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658362    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658372    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658387    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658414    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658459    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 14:56:33.658635 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.658471    4470 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 14:56:33.662046 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.662028    4470 server.go:1263] "Started kubelet"
Nov 02 14:56:33.662158 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.662082    4470 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 14:56:33.662223 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.662144    4470 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 14:56:33.662279 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.662238    4470 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 14:56:33.662569 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.662549    4470 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 14:56:33.662611 ip-10-0-139-197 systemd[1]: Started Kubernetes Kubelet.
Nov 02 14:56:33.664698 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.664682    4470 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 14:56:33.664875 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.664860    4470 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 14:56:33.664875 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.664870    4470 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.665149    4470 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 14:50:55 +0000 UTC" deadline="2028-06-19 22:14:21.89887568 +0000 UTC"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.665199    4470 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14287h17m48.233693532s"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.665256    4470 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.665264    4470 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.665261    4470 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:56:33.666591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.665514    4470 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 14:56:33.668091 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668047    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e573dd1-b81f-4134-9454-739323385c26" volumeName="kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 14:56:33.668180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668099    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3" volumeName="kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca" seLinuxMountContext=""
Nov 02 14:56:33.668180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668115    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f641dd2a-3e2d-474c-a777-758342eebfb0" volumeName="kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4" seLinuxMountContext=""
Nov 02 14:56:33.668180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668131    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d5ec2fd-9039-46e6-924e-9f60195c478b" volumeName="kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p" seLinuxMountContext=""
Nov 02 14:56:33.668180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668152    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3" volumeName="kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs" seLinuxMountContext=""
Nov 02 14:56:33.668180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668166    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" volumeName="kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 14:56:33.668470 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668184    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed89b459-a1e3-4cb7-a05c-1e79ba406239" volumeName="kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c" seLinuxMountContext=""
Nov 02 14:56:33.668470 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668219    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffe9e011-abd5-4da2-a109-2eb26c91e182" volumeName="kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy" seLinuxMountContext=""
Nov 02 14:56:33.668470 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668438    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e7beaa44-267d-4887-a21d-077834e70625" volumeName="kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d" seLinuxMountContext=""
Nov 02 14:56:33.668628 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668579    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f641dd2a-3e2d-474c-a777-758342eebfb0" volumeName="kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key" seLinuxMountContext=""
Nov 02 14:56:33.668628 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668600    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd0c7197-ef21-4d95-94bc-133fbd54c166" volumeName="kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww" seLinuxMountContext=""
Nov 02 14:56:33.668730 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668625    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d80c35b-aefd-4e05-b331-f30b815e11e6" volumeName="kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume" seLinuxMountContext=""
Nov 02 14:56:33.668782 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668735    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cac903e-68cd-473b-9177-e0acbcbee02a" volumeName="kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6" seLinuxMountContext=""
Nov 02 14:56:33.668782 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668756    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e573dd1-b81f-4134-9454-739323385c26" volumeName="kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides" seLinuxMountContext=""
Nov 02 14:56:33.668889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668783    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f641dd2a-3e2d-474c-a777-758342eebfb0" volumeName="kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle" seLinuxMountContext=""
Nov 02 14:56:33.668961 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668900    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="072126f8-4f44-46b1-a1ca-221a57d7feb3" volumeName="kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4" seLinuxMountContext=""
Nov 02 14:56:33.668961 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668919    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="441ad735-6cb4-45c4-a3a0-cdde58b98a47" volumeName="kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h" seLinuxMountContext=""
Nov 02 14:56:33.669062 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668968    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d80c35b-aefd-4e05-b331-f30b815e11e6" volumeName="kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7" seLinuxMountContext=""
Nov 02 14:56:33.669062 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.668989    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" volumeName="kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 14:56:33.669062 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669004    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb9bfe09-6645-4ba6-bdbb-62a02364fbb1" volumeName="kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret" seLinuxMountContext=""
Nov 02 14:56:33.669253 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669119    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffe9e011-abd5-4da2-a109-2eb26c91e182" volumeName="kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config" seLinuxMountContext=""
Nov 02 14:56:33.669253 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669137    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffe9e011-abd5-4da2-a109-2eb26c91e182" volumeName="kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw" seLinuxMountContext=""
Nov 02 14:56:33.669253 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669154    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="072126f8-4f44-46b1-a1ca-221a57d7feb3" volumeName="kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume" seLinuxMountContext=""
Nov 02 14:56:33.669391 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669174    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6441f0c0-0287-4d61-bbb6-ac5268f691e3" volumeName="kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9" seLinuxMountContext=""
Nov 02 14:56:33.669433 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669392    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84352a64-ea29-427b-a1bb-93bd7aa3298a" volumeName="kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9" seLinuxMountContext=""
Nov 02 14:56:33.669433 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669415    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e573dd1-b81f-4134-9454-739323385c26" volumeName="kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5" seLinuxMountContext=""
Nov 02 14:56:33.669498 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669431    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" volumeName="kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy" seLinuxMountContext=""
Nov 02 14:56:33.669498 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669458    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84352a64-ea29-427b-a1bb-93bd7aa3298a" volumeName="kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script" seLinuxMountContext=""
Nov 02 14:56:33.669498 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669485    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" volumeName="kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7" seLinuxMountContext=""
Nov 02 14:56:33.669584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669505    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bd0c7197-ef21-4d95-94bc-133fbd54c166" volumeName="kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca" seLinuxMountContext=""
Nov 02 14:56:33.669584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669522    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="072126f8-4f44-46b1-a1ca-221a57d7feb3" volumeName="kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir" seLinuxMountContext=""
Nov 02 14:56:33.669584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669537    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6441f0c0-0287-4d61-bbb6-ac5268f691e3" volumeName="kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert" seLinuxMountContext=""
Nov 02 14:56:33.669584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669558    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cac903e-68cd-473b-9177-e0acbcbee02a" volumeName="kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned" seLinuxMountContext=""
Nov 02 14:56:33.669584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669577    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e573dd1-b81f-4134-9454-739323385c26" volumeName="kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669596    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95" volumeName="kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8" seLinuxMountContext=""
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669610    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed89b459-a1e3-4cb7-a05c-1e79ba406239" volumeName="kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir" seLinuxMountContext=""
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669624    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8cac903e-68cd-473b-9177-e0acbcbee02a" volumeName="kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp" seLinuxMountContext=""
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669642    4470 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e573dd1-b81f-4134-9454-739323385c26" volumeName="kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config" seLinuxMountContext=""
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669655    4470 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 14:56:33.669757 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.669664    4470 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 14:56:33.671174 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.671152    4470 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 14:56:33.671174 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.671175    4470 factory.go:55] Registering systemd factory
Nov 02 14:56:33.671350 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.671204    4470 factory.go:223] Registration of the systemd container factory successfully
Nov 02 14:56:33.672552 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.672530    4470 factory.go:153] Registering CRI-O factory
Nov 02 14:56:33.672633 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.672555    4470 factory.go:223] Registration of the crio container factory successfully
Nov 02 14:56:33.672633 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.672578    4470 factory.go:103] Registering Raw factory
Nov 02 14:56:33.672633 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.672612    4470 manager.go:1199] Started watching for new ooms in manager
Nov 02 14:56:33.673174 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.673161    4470 manager.go:319] Starting recovery of all containers
Nov 02 14:56:33.681616 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.681591    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:33.682301 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.682278    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:33.696811 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.696776    4470 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 14:56:33.698046 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.698021    4470 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 14:56:33.698176 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.698052    4470 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 14:56:33.698176 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.698076    4470 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 14:56:33.698176 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.698120    4470 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 14:56:33.706652 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706609    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="2015146dcf90e484ad2d0e2f1e9e3ff0c3f35f4cf6a1c2910fe9be48a839a18e" exitCode=0
Nov 02 14:56:33.706652 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706650    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="1fbf7ea81184277c01017f730cd1aa6d3475df3ef190e831797a61be2ba8b9c0" exitCode=0
Nov 02 14:56:33.706819 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706662    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="9ca197b4df5e335ef6d80c4e378f49d5d17665a68c1d90b135dda9f5c2776743" exitCode=0
Nov 02 14:56:33.706819 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706673    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="55fca286f05a95f75f51dae49157c2b701bd3e3a1919405bb25669d532bf42d9" exitCode=0
Nov 02 14:56:33.706819 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706683    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="c4f4a741c1e8beeb8b9b98c678329cddb64cc70d84f92933d2dd01c09627db5b" exitCode=0
Nov 02 14:56:33.706819 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.706692    4470 generic.go:356] "Generic (PLEG): container finished" podID="bc1c4566-cd90-43ea-b3cb-6f4b96631b9b" containerID="9dd013b68ca04b66d13824236dde05e3b273552060aa92a4e7711eae91153a94" exitCode=0
Nov 02 14:56:33.732059 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.732029    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 14:56:33.732413 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.732384    4470 generic.go:356] "Generic (PLEG): container finished" podID="8e573dd1-b81f-4134-9454-739323385c26" containerID="482381b200dc4d254cf8f3737f12e325b8e1be1b643277ec99731cabb40e84dc" exitCode=1
Nov 02 14:56:33.736046 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.736014    4470 generic.go:356] "Generic (PLEG): container finished" podID="fd479be79ef05ee5c1afc91f51096ca7" containerID="5c9c874f50dfd5ab684cb69e14a75d48a326bc814f79d9f1ad0c59d54528503d" exitCode=0
Nov 02 14:56:33.739040 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.739024    4470 manager.go:324] Recovery completed
Nov 02 14:56:33.754662 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.754635    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.755941 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.755924    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.756033 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.755959    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.756033 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.755973    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.757490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757474    4470 cpu_manager.go:226] "Starting" policy="none"
Nov 02 14:56:33.757490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757485    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.757490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757489    4470 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 14:56:33.757592 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757506    4470 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 14:56:33.757639 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757625    4470 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757640    4470 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757657    4470 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757662    4470 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757668    4470 policy_none.go:50] "Start"
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757676    4470 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 14:56:33.757693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.757684    4470 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 14:56:33.758395 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.758375    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.758513 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.758411    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.758513 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.758427    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.766144 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.766116    4470 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:56:33.788808 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.788778    4470 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 14:56:33.788808 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.788810    4470 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 14:56:33.789020 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.788821    4470 policy_none.go:44] "Start"
Nov 02 14:56:33.793172 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.793137    4470 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 14:56:33.795814 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.795800    4470 manager.go:341] "Starting Device Plugin manager"
Nov 02 14:56:33.795906 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.795834    4470 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 14:56:33.795906 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.795844    4470 server.go:87] "Starting device plugin registration server"
Nov 02 14:56:33.796199 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796174    4470 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 14:56:33.796276 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796204    4470 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 14:56:33.796348 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796325    4470 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 14:56:33.796485 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796442    4470 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 14:56:33.796557 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796487    4470 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 14:56:33.796557 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796494    4470 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 14:56:33.796647 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.796552    4470 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T14:56:33.79647543Z","UUID":"e1f063d4-d214-471d-a62f-afc14a710606","Handler":null,"Name":"","Endpoint":""}
Nov 02 14:56:33.797802 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.797776    4470 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 14:56:33.797894 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.797828    4470 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-197.us-west-1.compute.internal\" not found"
Nov 02 14:56:33.798064 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.798045    4470 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 14:56:33.798162 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.798077    4470 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 14:56:33.798245 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.798211    4470 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"]
Nov 02 14:56:33.798367 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.798353    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.799524 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.799507    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.799630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.799537    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.799630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.799553    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.799882 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.799866    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.800105 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.800085    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.801152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801136    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.801260 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801166    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.801260 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801179    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.801260 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801137    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.801260 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801239    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.801260 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801253    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.801469 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.801344    4470 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.801548 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801520    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"6e7f87aa5e9c9e37d740f6ee1d873224","Type":"ContainerStarted","Data":"93860044e1aa061004f22e10c16462cb9ea8ac896532330474e729dbef75e562"}
Nov 02 14:56:33.801585 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801566    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"6e7f87aa5e9c9e37d740f6ee1d873224","Type":"ContainerStarted","Data":"551f373508e6eaa97f1db26f49b4c147db2945db8b826449774c614e1974282c"}
Nov 02 14:56:33.801640 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801627    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerStarted","Data":"cb68004caf94874b8cf3f091a6abdfad7ac8ae574260844adef7ba1cbd200b7e"}
Nov 02 14:56:33.801686 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801649    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerDied","Data":"5c9c874f50dfd5ab684cb69e14a75d48a326bc814f79d9f1ad0c59d54528503d"}
Nov 02 14:56:33.801686 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801666    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal" event={"ID":"fd479be79ef05ee5c1afc91f51096ca7","Type":"ContainerStarted","Data":"3826635986bf81a28bfece71671de5ab0c2156133f673ae6120d29d75bbc68ed"}
Nov 02 14:56:33.801785 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.801710    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.802619 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.802602    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.802713 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.802635    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.802713 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.802649    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.802823 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:33.802802    4470 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-197.us-west-1.compute.internal\" not found" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.866603 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.866566    4470 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 14:56:33.867596 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.867580    4470 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:56:33.896626 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.896595    4470 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 14:56:33.897748 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.897729    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 14:56:33.897858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.897761    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 14:56:33.897858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.897771    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 14:56:33.897858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.897807    4470 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.917032 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.916773    4470 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:56:33.918631 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.917263    4470 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:56:33.918631 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.917463    4470 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 14:56:33.966916 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.966873    4470 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.966987 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.966934    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.966987 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.966966    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:33.967075 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:33.966993    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.009822 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.009791    4470 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.009935 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.009870    4470 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.010465 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.010450    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeNotReady"
Nov 02 14:56:34.010516 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.010481    4470 setters.go:546] "Node became not ready" node="ip-10-0-139-197.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T14:56:34Z","lastTransitionTime":"2026-11-02T14:56:34Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 14:56:34.028402 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.028375    4470 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 14:56:34.028472 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.028441    4470 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.028472 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.028461    4470 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.061862 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.061823    4470 nodeinfomanager.go:471] existing CSINode "ip-10-0-139-197.us-west-1.compute.internal" is owned by different node (oldNodeID="5cb3d1da-3852-48a5-9edd-54a0c2f93877", newNodeID=""), cleaning up...
Nov 02 14:56:34.067531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067505    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.067612 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067548    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.067612 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067583    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.067707 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067605    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.067707 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067632    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6e7f87aa5e9c9e37d740f6ee1d873224-config\") pod \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"6e7f87aa5e9c9e37d740f6ee1d873224\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.067707 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.067672    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd479be79ef05ee5c1afc91f51096ca7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" (UID: \"fd479be79ef05ee5c1afc91f51096ca7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.085713 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.085684    4470 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 14:56:34.085852 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.085731    4470 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.648434 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.648398    4470 apiserver.go:52] "Watching apiserver"
Nov 02 14:56:34.738650 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.738622    4470 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.757213 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.757175    4470 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 14:56:34.757686 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.757649    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vsttp","openshift-dns/dns-default-5t56l","openshift-dns/node-resolver-xrvk7","openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl","openshift-cluster-node-tuning-operator/tuned-sz7rm","openshift-image-registry/node-ca-ldxkl","openshift-multus/network-metrics-daemon-z6trp","openshift-multus/multus-d59wq","openshift-network-diagnostics/network-check-target-g5d86","openshift-network-operator/iptables-alerter-qs4d5","openshift-multus/multus-additional-cni-plugins-hl98t","openshift-insights/insights-runtime-extractor-fjzx9","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal","openshift-ingress-canary/ingress-canary-cqlqj","openshift-service-ca/service-ca-5fffc97bdf-g8pcw","kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal","openshift-ovn-kubernetes/ovnkube-node-kndjb","kube-system/global-pull-secret-syncer-d7qbm"]
Nov 02 14:56:34.759642 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.759622    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:34.762438 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.762413    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.763178 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.763159    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.767925 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.767901    4470 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 14:56:34.770940 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.770923    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.770993 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.770950    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.770993 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.770970    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.770993 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.770986    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.771109 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771011    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.771109 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771036    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.771109 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771060    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.771109 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771094    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d80c35b-aefd-4e05-b331-f30b815e11e6-data-volume\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.771252 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771169    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771252 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771241    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771323 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771269    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771323 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771288    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.771323 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771313    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.771435 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771349    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7t6\" (UniqueName: \"kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.771435 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771367    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771435 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771388    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.771435 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771406    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts97h\" (UniqueName: \"kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:34.771435 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771421    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.771644 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771440    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.771644 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771588    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.771644 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771612    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.771644 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771636    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771771 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771656    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.771771 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771679    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.771771 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771703    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.771771 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771722    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.771907 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771784    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.771907 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771809    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvdz5\" (UniqueName: \"kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.771907 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771832    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771910    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771951    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.771986    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772007    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772026    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772048    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772074    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.772132 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772112    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772210    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772238    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772256    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772288    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8z7\" (UniqueName: \"kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772309    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772325    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772357    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772375    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772424    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z84s7\" (UniqueName: \"kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772435    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/072126f8-4f44-46b1-a1ca-221a57d7feb3-tmp-dir\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.772450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772447    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772470    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772484    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed89b459-a1e3-4cb7-a05c-1e79ba406239-tmp-dir\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772548    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772572    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772623    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772648    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772671    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772696    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772755    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772805    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772837    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n249p\" (UniqueName: \"kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p\") pod \"migrator-66df5d496b-4lshl\" (UID: \"8d5ec2fd-9039-46e6-924e-9f60195c478b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:34.772858 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772861    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772899    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.772963    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvz6c\" (UniqueName: \"kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773004    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773093    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773120    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773149    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773234    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773261    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.773305 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773279    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773295    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773328    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773348    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-tmp\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773344    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773403    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9bb9\" (UniqueName: \"kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773531    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.773570 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773559    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.773762 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773580    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdlh4\" (UniqueName: \"kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.773762 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773627    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.773762 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773659    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztmd9\" (UniqueName: \"kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.773762 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773691    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.773968 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773865    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zpb8\" (UniqueName: \"kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.773968 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773940    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.774081 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.773969    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.774081 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774009    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774081 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774034    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774081 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774053    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxfqw\" (UniqueName: \"kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774094    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774126    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774145    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-tuned\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774153    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd7ww\" (UniqueName: \"kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774209    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.774270 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774242    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.774459 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774270    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774459 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774296    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774459 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774321    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.774459 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774354    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.774459 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774397    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2jv4\" (UniqueName: \"kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.774606 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774513    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.774606 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774562    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.774606 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774583    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.774606 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.774603    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.799765 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.799739    4470 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 14:56:34.799876 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.799792    4470 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-197.us-west-1.compute.internal"
Nov 02 14:56:34.812061 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.812032    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rrdxh\""
Nov 02 14:56:34.812061 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.812043    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8dn6m\""
Nov 02 14:56:34.812256 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.812160    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-ptvxg\""
Nov 02 14:56:34.812976 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.812959    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.812976 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.812966    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.813184 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.813170    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.813252 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.813200    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 14:56:34.813252 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.813199    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.822915 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.822893    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.849772 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.849737    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8z7\" (UniqueName: \"kubernetes.io/projected/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-kube-api-access-5p8z7\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.850501 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.850479    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts97h\" (UniqueName: \"kubernetes.io/projected/441ad735-6cb4-45c4-a3a0-cdde58b98a47-kube-api-access-ts97h\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:34.855713 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.855692    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz46d\" (UniqueName: \"kubernetes.io/projected/e7beaa44-267d-4887-a21d-077834e70625-kube-api-access-vz46d\") pod \"network-check-target-g5d86\" (UID: \"e7beaa44-267d-4887-a21d-077834e70625\") " pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:34.864078 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.864054    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxfqw\" (UniqueName: \"kubernetes.io/projected/ffe9e011-abd5-4da2-a109-2eb26c91e182-kube-api-access-bxfqw\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875072 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875043    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875140 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875091    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.875172 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875152    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875225 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875179    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875266 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875230    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875266 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875241    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-systemd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875266 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875241    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-socket-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875243    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-kubelet-config\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875277    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-conf-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875290    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-host\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875255    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875325    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-registration-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875339    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875365    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875388    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-dbus\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875390    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875426    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875433    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-node-log\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875436 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875437    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875466    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875477    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-etc-kubernetes\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875508    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875514    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-device-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875543    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875561    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875567    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875590    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-multus-certs\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875590    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875623    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875635    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-var-lib-kubelet\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875647    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875652    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-run\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875665    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875673    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-systemd-units\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875689    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875693    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-netd\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875734    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875759    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875790    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875797    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875817    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875832    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed89b459-a1e3-4cb7-a05c-1e79ba406239-hosts-file\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875843    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.875857 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875867    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-log-socket\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875876    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-cni-bin\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875870    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875835    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875911    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875913    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-system-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875940    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875964    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875985    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.875963    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-kubelet\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876009    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876028    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-socket-dir-parent\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876071    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876081    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-systemd\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876031    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-sys\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876105    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-netns\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876115    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876138    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876142    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84352a64-ea29-427b-a1bb-93bd7aa3298a-host-slash\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876162    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876178    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876215    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876218    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d80c35b-aefd-4e05-b331-f30b815e11e6-crio-socket\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876216    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-slash\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876253    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cnibin\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876252    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876285    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876308    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876338    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-system-cni-dir\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876342    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876368    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-run-k8s-cni-cncf-io\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876389    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-cni-dir\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876400    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd0c7197-ef21-4d95-94bc-133fbd54c166-host\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876427    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876373    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-multus\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876466    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876484    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876496    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysconfig\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876500    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876541    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876544    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-cnibin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876567    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876596    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-lib-modules\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876605    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-run-ovn\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876632    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-modprobe-d\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876630    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876653    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.876645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876656    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876677    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-host-var-lib-cni-bin\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876678    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876705    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ffe9e011-abd5-4da2-a109-2eb26c91e182-hostroot\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876708    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876724    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876739    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876765    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876786    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876806    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-run-netns\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876808    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876822    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-etc-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876844    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876849    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-etc-selinux\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876865    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876867    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-sysctl-conf\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876875    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-sys-fs\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876894    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-host-kubelet\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876913    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876927    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8e573dd1-b81f-4134-9454-739323385c26-var-lib-openvswitch\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876950    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:34.877864 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.876977    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cac903e-68cd-473b-9177-e0acbcbee02a-etc-kubernetes\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.911034 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.910956    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 14:56:34.911152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911099    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 02 14:56:34.911397 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911374    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 14:56:34.911513 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911494    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fbnz4\""
Nov 02 14:56:34.911638 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911598    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zslfv\""
Nov 02 14:56:34.911638 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911607    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 14:56:34.911638 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911627    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5lvgl\""
Nov 02 14:56:34.911917 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.911892    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.912276 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912202    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 14:56:34.912276 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912202    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bm8rt\""
Nov 02 14:56:34.912276 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912220    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 14:56:34.912276 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912272    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 14:56:34.912571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912473    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.912571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912504    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 14:56:34.912571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912509    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.912571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912512    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mcvdb\""
Nov 02 14:56:34.912571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912559    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912600    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912609    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912613    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912601    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912682    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6441f0c0-0287-4d61-bbb6-ac5268f691e3-cert\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912734    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912740    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 14:56:34.912798 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912797    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-5w9z9\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912807    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zct6j\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912817    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-dh847\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912833    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912835    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dhf26\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912808    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-zp9vx\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.912857    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m4lkk\""
Nov 02 14:56:34.913025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913009    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913035    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913045    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913053    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913067    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913069    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913099    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-snsjj\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913121    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913141    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913143    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-agent-certs\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913156    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913168    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-key\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913204    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913228    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nw765\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913240    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913268    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913289    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913297    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913307    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913311 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913311    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-config\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913350    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913375    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913500    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913505    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-wtdgk\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913566    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913580    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e573dd1-b81f-4134-9454-739323385c26-ovn-node-metrics-cert\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913598    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913640    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913697    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913878    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd7ww\" (UniqueName: \"kubernetes.io/projected/bd0c7197-ef21-4d95-94bc-133fbd54c166-kube-api-access-sd7ww\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913877    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztmd9\" (UniqueName: \"kubernetes.io/projected/84352a64-ea29-427b-a1bb-93bd7aa3298a-kube-api-access-ztmd9\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.913991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.913960    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2jv4\" (UniqueName: \"kubernetes.io/projected/f641dd2a-3e2d-474c-a777-758342eebfb0-kube-api-access-v2jv4\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.914352 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.914307    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zpb8\" (UniqueName: \"kubernetes.io/projected/9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95-kube-api-access-8zpb8\") pod \"aws-ebs-csi-driver-node-gjc74\" (UID: \"9e3b4ac6-1c86-4cff-88e6-b3789fc6fe95\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gjc74"
Nov 02 14:56:34.914454 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.914437    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-env-overrides\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.914549 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.914535    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.914578 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.914553    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8e573dd1-b81f-4134-9454-739323385c26-ovnkube-script-lib\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.914690 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.914675    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-cni-binary-copy\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.916848 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.916831    4470 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:34.916921 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.916833    4470 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:34.916971 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.916956    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:35.416926566 +0000 UTC m=+1.854828702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:34.917029 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.917003    4470 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:34.917029 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.917010    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:35.416995321 +0000 UTC m=+1.854897442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:34.917115 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.917044    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:35.417032951 +0000 UTC m=+1.854935288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:34.917115 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.917070    4470 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 14:56:34.917205 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:34.917125    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:35.417115021 +0000 UTC m=+1.855017140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : secret "metrics-daemon-secret" not found
Nov 02 14:56:34.920780 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920753    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z84s7\" (UniqueName: \"kubernetes.io/projected/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-api-access-z84s7\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:34.920890 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920843    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7t6\" (UniqueName: \"kubernetes.io/projected/8cac903e-68cd-473b-9177-e0acbcbee02a-kube-api-access-9x7t6\") pod \"tuned-sz7rm\" (UID: \"8cac903e-68cd-473b-9177-e0acbcbee02a\") " pod="openshift-cluster-node-tuning-operator/tuned-sz7rm"
Nov 02 14:56:34.920890 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920841    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdlh4\" (UniqueName: \"kubernetes.io/projected/072126f8-4f44-46b1-a1ca-221a57d7feb3-kube-api-access-tdlh4\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.920890 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920859    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9bb9\" (UniqueName: \"kubernetes.io/projected/6441f0c0-0287-4d61-bbb6-ac5268f691e3-kube-api-access-r9bb9\") pod \"ingress-canary-cqlqj\" (UID: \"6441f0c0-0287-4d61-bbb6-ac5268f691e3\") " pod="openshift-ingress-canary/ingress-canary-cqlqj"
Nov 02 14:56:34.920890 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920845    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n249p\" (UniqueName: \"kubernetes.io/projected/8d5ec2fd-9039-46e6-924e-9f60195c478b-kube-api-access-n249p\") pod \"migrator-66df5d496b-4lshl\" (UID: \"8d5ec2fd-9039-46e6-924e-9f60195c478b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-4lshl"
Nov 02 14:56:34.921025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920912    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvz6c\" (UniqueName: \"kubernetes.io/projected/ed89b459-a1e3-4cb7-a05c-1e79ba406239-kube-api-access-tvz6c\") pod \"node-resolver-xrvk7\" (UID: \"ed89b459-a1e3-4cb7-a05c-1e79ba406239\") " pod="openshift-dns/node-resolver-xrvk7"
Nov 02 14:56:34.921025 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.920934    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvdz5\" (UniqueName: \"kubernetes.io/projected/8e573dd1-b81f-4134-9454-739323385c26-kube-api-access-hvdz5\") pod \"ovnkube-node-kndjb\" (UID: \"8e573dd1-b81f-4134-9454-739323385c26\") " pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:34.921762 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.921744    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd0c7197-ef21-4d95-94bc-133fbd54c166-serviceca\") pod \"node-ca-ldxkl\" (UID: \"bd0c7197-ef21-4d95-94bc-133fbd54c166\") " pod="openshift-image-registry/node-ca-ldxkl"
Nov 02 14:56:34.921828 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.921798    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/072126f8-4f44-46b1-a1ca-221a57d7feb3-config-volume\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:34.921828 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.921808    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bc1c4566-cd90-43ea-b3cb-6f4b96631b9b-cni-binary-copy\") pod \"multus-additional-cni-plugins-hl98t\" (UID: \"bc1c4566-cd90-43ea-b3cb-6f4b96631b9b\") " pod="openshift-multus/multus-additional-cni-plugins-hl98t"
Nov 02 14:56:34.921974 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.921957    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fb9bfe09-6645-4ba6-bdbb-62a02364fbb1-original-pull-secret\") pod \"global-pull-secret-syncer-d7qbm\" (UID: \"fb9bfe09-6645-4ba6-bdbb-62a02364fbb1\") " pod="kube-system/global-pull-secret-syncer-d7qbm"
Nov 02 14:56:34.922970 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.922953    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f641dd2a-3e2d-474c-a777-758342eebfb0-signing-cabundle\") pod \"service-ca-5fffc97bdf-g8pcw\" (UID: \"f641dd2a-3e2d-474c-a777-758342eebfb0\") " pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw"
Nov 02 14:56:34.923062 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.923047    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/84352a64-ea29-427b-a1bb-93bd7aa3298a-iptables-alerter-script\") pod \"iptables-alerter-qs4d5\" (UID: \"84352a64-ea29-427b-a1bb-93bd7aa3298a\") " pod="openshift-network-operator/iptables-alerter-qs4d5"
Nov 02 14:56:34.923107 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.923061    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ffe9e011-abd5-4da2-a109-2eb26c91e182-multus-daemon-config\") pod \"multus-d59wq\" (UID: \"ffe9e011-abd5-4da2-a109-2eb26c91e182\") " pod="openshift-multus/multus-d59wq"
Nov 02 14:56:34.924170 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:34.924155    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3-konnectivity-ca\") pod \"konnectivity-agent-vsttp\" (UID: \"ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3\") " pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:35.067103 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.067081    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 14:56:35.385068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.385040    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:35.481139 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.481098    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:35.481139 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.481145    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:35.481336 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481256    4470 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:35.481336 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481261    4470 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:35.481336 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.481278    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:35.481336 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481311    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:36.481294916 +0000 UTC m=+2.919197038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:35.481472 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481341    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:36.481332254 +0000 UTC m=+2.919234377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:35.481472 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481362    4470 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:35.481472 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:35.481367    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:35.481472 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481417    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:36.481405159 +0000 UTC m=+2.919307292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:35.481472 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481457    4470 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 14:56:35.481615 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:35.481514    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:36.481501856 +0000 UTC m=+2.919403980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : secret "metrics-daemon-secret" not found
Nov 02 14:56:36.292676 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.292646    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:36.488741 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.488704    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:36.488934 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.488751    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:36.488934 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.488803    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:36.488934 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.488833    4470 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:36.488934 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.488839    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:36.488934 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.488896    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:38.488878648 +0000 UTC m=+4.926780769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.488950    4470 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.488964    4470 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.488989    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:38.488978294 +0000 UTC m=+4.926880428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.489034    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:38.489016399 +0000 UTC m=+4.926918523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : secret "metrics-daemon-secret" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.489103    4470 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:36.489173 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:36.489157    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:38.489144902 +0000 UTC m=+4.927047025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:36.743918 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:36.743885    4470 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 14:56:37.846894 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:37.846863    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xrvk7_ed89b459-a1e3-4cb7-a05c-1e79ba406239/dns-node-resolver/0.log"
Nov 02 14:56:38.506243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.506184    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.506259    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.506291    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506302    4470 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.506312    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506370    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:42.506355121 +0000 UTC m=+8.944257241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506388    4470 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506409    4470 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506435    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs podName:441ad735-6cb4-45c4-a3a0-cdde58b98a47 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:42.506422639 +0000 UTC m=+8.944324760 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs") pod "network-metrics-daemon-z6trp" (UID: "441ad735-6cb4-45c4-a3a0-cdde58b98a47") : secret "metrics-daemon-secret" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506406    4470 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506461    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:42.506446616 +0000 UTC m=+8.944348740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:38.506526 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:38.506480    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:42.506470129 +0000 UTC m=+8.944372253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:38.534182 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.534153    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kndjb"
Nov 02 14:56:38.749970 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.749942    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-g8pcw" event={"ID":"f641dd2a-3e2d-474c-a777-758342eebfb0","Type":"ContainerStarted","Data":"8fa86d9fcf3d6708a62f02ebe9c5a7053bfa0881a5bb0acef76b802977529b39"}
Nov 02 14:56:38.847118 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:38.847045    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ldxkl_bd0c7197-ef21-4d95-94bc-133fbd54c166/node-ca/0.log"
Nov 02 14:56:39.249451 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:39.249409    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cqlqj_6441f0c0-0287-4d61-bbb6-ac5268f691e3/serve-healthcheck-canary/0.log"
Nov 02 14:56:39.847376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:39.847348    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-4lshl_8d5ec2fd-9039-46e6-924e-9f60195c478b/migrator/0.log"
Nov 02 14:56:41.661546 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:41.661507    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:56:42.536568 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.536529    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:42.536756 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.536579    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:42.536756 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.536622    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:42.536756 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.536650    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:42.536756 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536688    4470 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 14:56:42.536888 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536759    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls podName:072126f8-4f44-46b1-a1ca-221a57d7feb3 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:50.536741891 +0000 UTC m=+16.974644025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls") pod "dns-default-5t56l" (UID: "072126f8-4f44-46b1-a1ca-221a57d7feb3") : secret "dns-default-metrics-tls" not found
Nov 02 14:56:42.536888 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536760    4470 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 14:56:42.536888 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536778    4470 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:42.536888 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536830    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:50.536809219 +0000 UTC m=+16.974711352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : configmap "kube-rbac-proxy" not found
Nov 02 14:56:42.536888 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:56:42.536849    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls podName:7d80c35b-aefd-4e05-b331-f30b815e11e6 nodeName:}" failed. No retries permitted until 2026-11-02 14:56:50.536841941 +0000 UTC m=+16.974744060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fjzx9" (UID: "7d80c35b-aefd-4e05-b331-f30b815e11e6") : secret "insights-runtime-extractor-tls" not found
Nov 02 14:56:42.537044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.537030    4470 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 14:56:42.539115 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.539083    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/441ad735-6cb4-45c4-a3a0-cdde58b98a47-metrics-certs\") pod \"network-metrics-daemon-z6trp\" (UID: \"441ad735-6cb4-45c4-a3a0-cdde58b98a47\") " pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:42.560071 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.560037    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-z6trp"
Nov 02 14:56:42.925679 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:42.925582    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-z6trp"]
Nov 02 14:56:42.929785 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:42.929755    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441ad735_6cb4_45c4_a3a0_cdde58b98a47.slice/crio-b5d86c465cd80b20375bb6f3d95b2598d197c91195e6f137bff44a8e54057952 WatchSource:0}: Error finding container b5d86c465cd80b20375bb6f3d95b2598d197c91195e6f137bff44a8e54057952: Status 404 returned error can't find the container with id b5d86c465cd80b20375bb6f3d95b2598d197c91195e6f137bff44a8e54057952
Nov 02 14:56:43.763153 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:43.763115    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6trp" event={"ID":"441ad735-6cb4-45c4-a3a0-cdde58b98a47","Type":"ContainerStarted","Data":"b5d86c465cd80b20375bb6f3d95b2598d197c91195e6f137bff44a8e54057952"}
Nov 02 14:56:44.448837 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:44.448812    4470 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-197.us-west-1.compute.internal" event="NodeReady"
Nov 02 14:56:44.770282 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:44.770244    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6trp" event={"ID":"441ad735-6cb4-45c4-a3a0-cdde58b98a47","Type":"ContainerStarted","Data":"b0163d1eba82f360272f1eca8db24196f3e0b5ac58c2f76522e2f17172daf6b1"}
Nov 02 14:56:44.770500 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:44.770320    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-z6trp" event={"ID":"441ad735-6cb4-45c4-a3a0-cdde58b98a47","Type":"ContainerStarted","Data":"254242eaa6028a9fe39bf11ccf90d8be945e9b296113ca95d1b01f4dc5fa3923"}
Nov 02 14:56:50.597884 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.597839    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:50.598449 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.597911    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:50.598449 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.597946    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:50.598449 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.598391    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d80c35b-aefd-4e05-b331-f30b815e11e6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:50.600222 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.600180    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d80c35b-aefd-4e05-b331-f30b815e11e6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fjzx9\" (UID: \"7d80c35b-aefd-4e05-b331-f30b815e11e6\") " pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:50.600322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.600226    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/072126f8-4f44-46b1-a1ca-221a57d7feb3-metrics-tls\") pod \"dns-default-5t56l\" (UID: \"072126f8-4f44-46b1-a1ca-221a57d7feb3\") " pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:50.663630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.663568    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:50.664627 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:50.664606    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fjzx9"
Nov 02 14:56:51.084529 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:51.084496    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5t56l"]
Nov 02 14:56:51.088350 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:51.088323    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072126f8_4f44_46b1_a1ca_221a57d7feb3.slice/crio-2cd803048ba55f1ec7f1c61e12735e46350f7bb40c838621ef1fa1a7848541ab WatchSource:0}: Error finding container 2cd803048ba55f1ec7f1c61e12735e46350f7bb40c838621ef1fa1a7848541ab: Status 404 returned error can't find the container with id 2cd803048ba55f1ec7f1c61e12735e46350f7bb40c838621ef1fa1a7848541ab
Nov 02 14:56:51.127091 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:51.127057    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d80c35b_aefd_4e05_b331_f30b815e11e6.slice/crio-8dcca89fe2091c5cb04185e8fb6a77922312bf5bf18cd0995b55beb9a89c4347 WatchSource:0}: Error finding container 8dcca89fe2091c5cb04185e8fb6a77922312bf5bf18cd0995b55beb9a89c4347: Status 404 returned error can't find the container with id 8dcca89fe2091c5cb04185e8fb6a77922312bf5bf18cd0995b55beb9a89c4347
Nov 02 14:56:51.135366 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:51.135277    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fjzx9"]
Nov 02 14:56:51.791542 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:51.791498    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5t56l" event={"ID":"072126f8-4f44-46b1-a1ca-221a57d7feb3","Type":"ContainerStarted","Data":"2cd803048ba55f1ec7f1c61e12735e46350f7bb40c838621ef1fa1a7848541ab"}
Nov 02 14:56:51.792929 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:51.792904    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjzx9" event={"ID":"7d80c35b-aefd-4e05-b331-f30b815e11e6","Type":"ContainerStarted","Data":"3b981421a960250c40b0f4320090736b9fbe5a0afb8e40c298309a56fd70c04e"}
Nov 02 14:56:51.792929 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:51.792952    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjzx9" event={"ID":"7d80c35b-aefd-4e05-b331-f30b815e11e6","Type":"ContainerStarted","Data":"8dcca89fe2091c5cb04185e8fb6a77922312bf5bf18cd0995b55beb9a89c4347"}
Nov 02 14:56:52.636428 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:52.636346    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vsttp"
Nov 02 14:56:52.798419 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:52.798385    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjzx9" event={"ID":"7d80c35b-aefd-4e05-b331-f30b815e11e6","Type":"ContainerStarted","Data":"be5ef93bd1e802c055d46a794f2a056ef0f3d90f44aba7caab3783a817e1d2f9"}
Nov 02 14:56:53.807917 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:53.807878    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5t56l" event={"ID":"072126f8-4f44-46b1-a1ca-221a57d7feb3","Type":"ContainerStarted","Data":"badc133f4de3bdd14ed02caf90d4ddb924485773afd1e534f497adbca7cab341"}
Nov 02 14:56:53.808405 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:53.807971    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5t56l" event={"ID":"072126f8-4f44-46b1-a1ca-221a57d7feb3","Type":"ContainerStarted","Data":"08ad00a8f29c9d1eb4169c4be7001d6c88d4b683a62537eb524c68db9383bf4b"}
Nov 02 14:56:53.808405 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:53.808062    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5t56l"
Nov 02 14:56:54.813313 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:54.813271    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fjzx9" event={"ID":"7d80c35b-aefd-4e05-b331-f30b815e11e6","Type":"ContainerStarted","Data":"37b7a7ad85b444602b7d60d9cc412f503a3b56d3d392d99afbdaafc11448e572"}
Nov 02 14:56:54.984722 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:54.984601    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"]
Nov 02 14:56:54.986827 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:54.986798    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:55.031207 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.031158    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/06b352aa-9fb8-4e3e-a4de-944438706944-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-zvdzm\" (UID: \"06b352aa-9fb8-4e3e-a4de-944438706944\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:55.039317 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.039291    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 02 14:56:55.039317 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.039292    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kxc47\""
Nov 02 14:56:55.046698 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.046673    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"]
Nov 02 14:56:55.131817 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.131729    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/06b352aa-9fb8-4e3e-a4de-944438706944-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-zvdzm\" (UID: \"06b352aa-9fb8-4e3e-a4de-944438706944\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:55.134075 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.134048    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/06b352aa-9fb8-4e3e-a4de-944438706944-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-zvdzm\" (UID: \"06b352aa-9fb8-4e3e-a4de-944438706944\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:55.296588 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.296549    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:55.661889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.661856    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"]
Nov 02 14:56:55.665665 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:55.665637    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b352aa_9fb8_4e3e_a4de_944438706944.slice/crio-3335695526ec35ae44e3afb62254e5df6bed708347e80cd9af2e622ab873f34f WatchSource:0}: Error finding container 3335695526ec35ae44e3afb62254e5df6bed708347e80cd9af2e622ab873f34f: Status 404 returned error can't find the container with id 3335695526ec35ae44e3afb62254e5df6bed708347e80cd9af2e622ab873f34f
Nov 02 14:56:55.817484 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:55.817451    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm" event={"ID":"06b352aa-9fb8-4e3e-a4de-944438706944","Type":"ContainerStarted","Data":"3335695526ec35ae44e3afb62254e5df6bed708347e80cd9af2e622ab873f34f"}
Nov 02 14:56:57.828159 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:57.828118    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm" event={"ID":"06b352aa-9fb8-4e3e-a4de-944438706944","Type":"ContainerStarted","Data":"f6830ddec19ffd8d819a0e97bfb1ed481cd008b83d51407e94e0551bd222475e"}
Nov 02 14:56:57.828677 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:57.828337    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:57.832837 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:57.832816    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm"
Nov 02 14:56:57.944273 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:57.944207    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-zvdzm" podStartSLOduration=2.562042233 podStartE2EDuration="3.944171045s" totalImagesPullingTime="1.382128812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:56:57.943571067 +0000 UTC m=+24.381473208" watchObservedRunningTime="2026-11-02 14:56:57.944171045 +0000 UTC m=+24.382073183"
Nov 02 14:56:58.865863 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.865827    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:56:58.868657 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.868634    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.920988 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.920956    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 02 14:56:58.958299 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958263    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958299 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958302    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958517 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958383    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958517 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958400    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxprr\" (UniqueName: \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958517 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958420    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958517 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958434    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.958647 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.958587    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:58.970591 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970551    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 02 14:56:58.970715 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970692    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 02 14:56:58.970769 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970730    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 02 14:56:58.970769 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970743    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tqnn7\""
Nov 02 14:56:58.970769 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970736    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 02 14:56:58.970895 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970798    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 02 14:56:58.970895 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:58.970837    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 02 14:56:59.020935 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.020898    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:56:59.059683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.059648    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.059683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.059683    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxprr\" (UniqueName: \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.059913 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.059707    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.059968 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.059946    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060020 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060009    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060069 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060057    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060121 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060076    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060613 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060588    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060791 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060769    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060852 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060787    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.060852 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.060771    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.062261 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.062234    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.062426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.062236    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.121462 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.121381    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxprr\" (UniqueName: \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\") pod \"console-59dcbcc7bd-h8grb\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") " pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.178607 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.178565    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:56:59.551622 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.551591    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:56:59.554745 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:56:59.554712    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60adcf0b_20e3_4abe_ba70_0d05ff446a11.slice/crio-a756b190158d2d151599d3c22652a53b6233b78659aff48eacf8ad55926a55b6 WatchSource:0}: Error finding container a756b190158d2d151599d3c22652a53b6233b78659aff48eacf8ad55926a55b6: Status 404 returned error can't find the container with id a756b190158d2d151599d3c22652a53b6233b78659aff48eacf8ad55926a55b6
Nov 02 14:56:59.591417 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.591390    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"]
Nov 02 14:56:59.593894 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.593877    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.646446 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.646416    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 02 14:56:59.646446 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.646415    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 14:56:59.660242 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.660221    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 02 14:56:59.660242 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.660221    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 14:56:59.660375 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.660236    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-87x8t\""
Nov 02 14:56:59.660375 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.660262    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 14:56:59.664507 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.664491    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.664585 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.664515    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5np8f\" (UniqueName: \"kubernetes.io/projected/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-kube-api-access-5np8f\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.664585 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.664541    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.664585 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.664561    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.696416 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.696383    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"]
Nov 02 14:56:59.765230 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.765171    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.765230 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.765236    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.765442 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.765281    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.765442 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.765299    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5np8f\" (UniqueName: \"kubernetes.io/projected/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-kube-api-access-5np8f\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.766057 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.766035    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.767813 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.767786    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.767861 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.767842    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.826637 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.826567    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5np8f\" (UniqueName: \"kubernetes.io/projected/d0c5373c-f6e3-47e5-adfc-7bb0496d9b37-kube-api-access-5np8f\") pod \"prometheus-operator-9cd89b74c-82lbh\" (UID: \"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:56:59.834974 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.834945    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcbcc7bd-h8grb" event={"ID":"60adcf0b-20e3-4abe-ba70-0d05ff446a11","Type":"ContainerStarted","Data":"a756b190158d2d151599d3c22652a53b6233b78659aff48eacf8ad55926a55b6"}
Nov 02 14:56:59.905052 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:56:59.905013    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"
Nov 02 14:57:00.273777 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:00.273744    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-82lbh"]
Nov 02 14:57:00.277474 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:00.277451    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0c5373c_f6e3_47e5_adfc_7bb0496d9b37.slice/crio-520dcc986aec37c693d644885564226263f587299f5f8fe4f48483a1ca41626d WatchSource:0}: Error finding container 520dcc986aec37c693d644885564226263f587299f5f8fe4f48483a1ca41626d: Status 404 returned error can't find the container with id 520dcc986aec37c693d644885564226263f587299f5f8fe4f48483a1ca41626d
Nov 02 14:57:00.839875 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:00.839829    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh" event={"ID":"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37","Type":"ContainerStarted","Data":"520dcc986aec37c693d644885564226263f587299f5f8fe4f48483a1ca41626d"}
Nov 02 14:57:02.848948 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:02.848903    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcbcc7bd-h8grb" event={"ID":"60adcf0b-20e3-4abe-ba70-0d05ff446a11","Type":"ContainerStarted","Data":"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"}
Nov 02 14:57:02.850852 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:02.850827    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh" event={"ID":"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37","Type":"ContainerStarted","Data":"37d772693876d64a327f50e31ca0107eb17baaf22d75a3e37ec28b40ad806496"}
Nov 02 14:57:02.850961 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:02.850872    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh" event={"ID":"d0c5373c-f6e3-47e5-adfc-7bb0496d9b37","Type":"ContainerStarted","Data":"f0253ac711a4bcff035592334a807e3424ad375af1b956a3efbd56edd64162f2"}
Nov 02 14:57:02.967686 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:02.967630    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59dcbcc7bd-h8grb" podStartSLOduration=1.879912818 podStartE2EDuration="4.967615962s" totalImagesPullingTime="3.087703144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:02.966728358 +0000 UTC m=+29.404630498" watchObservedRunningTime="2026-11-02 14:57:02.967615962 +0000 UTC m=+29.405518097"
Nov 02 14:57:03.086755 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:03.086689    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-82lbh" podStartSLOduration=2.330136982 podStartE2EDuration="4.086674522s" totalImagesPullingTime="1.75653754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:03.086210453 +0000 UTC m=+29.524112585" watchObservedRunningTime="2026-11-02 14:57:03.086674522 +0000 UTC m=+29.524576659"
Nov 02 14:57:03.563342 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:57:03.563303    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 14:57:03.815553 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:03.815477    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5t56l"
Nov 02 14:57:06.164375 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.164336    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bvwkz"]
Nov 02 14:57:06.189837 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.189808    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.291788 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.291747    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 14:57:06.314975 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.314945    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-wtmp\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.314975 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.314978    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-sys\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315236 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315010    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-tls\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315236 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315058    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-root\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315236 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315138    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-textfile\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315253    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315286    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdzj\" (UniqueName: \"kubernetes.io/projected/6f1f75d6-4610-427e-a5a1-0396c925b1ba-kube-api-access-7jdzj\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315332    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-metrics-client-ca\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.315353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.315350    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.341553 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.341516    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 14:57:06.341706 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.341599    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zx8m7\""
Nov 02 14:57:06.355422 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.355397    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 14:57:06.416098 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.415995    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-wtmp\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416098 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416044    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-sys\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416098 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416080    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-tls\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416098 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416102    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-root\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416119    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-textfile\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416141    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416245    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-wtmp\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416277    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-sys\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416243    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f1f75d6-4610-427e-a5a1-0396c925b1ba-root\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416301    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdzj\" (UniqueName: \"kubernetes.io/projected/6f1f75d6-4610-427e-a5a1-0396c925b1ba-kube-api-access-7jdzj\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416350    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-metrics-client-ca\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416381    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416860 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416594    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-textfile\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416860 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416795    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-accelerators-collector-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.416860 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.416835    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f75d6-4610-427e-a5a1-0396c925b1ba-metrics-client-ca\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.418631 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.418600    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-tls\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.418733 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.418697    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f1f75d6-4610-427e-a5a1-0396c925b1ba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.477084 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.477051    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdzj\" (UniqueName: \"kubernetes.io/projected/6f1f75d6-4610-427e-a5a1-0396c925b1ba-kube-api-access-7jdzj\") pod \"node-exporter-bvwkz\" (UID: \"6f1f75d6-4610-427e-a5a1-0396c925b1ba\") " pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.540291 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.540251    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bvwkz"
Nov 02 14:57:06.577377 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:06.577342    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1f75d6_4610_427e_a5a1_0396c925b1ba.slice/crio-fe398531d6472510553ae6a537e34e2973e6bef66edf7fd74c0c40db4ebb1626 WatchSource:0}: Error finding container fe398531d6472510553ae6a537e34e2973e6bef66edf7fd74c0c40db4ebb1626: Status 404 returned error can't find the container with id fe398531d6472510553ae6a537e34e2973e6bef66edf7fd74c0c40db4ebb1626
Nov 02 14:57:06.866324 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:06.866284    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvwkz" event={"ID":"6f1f75d6-4610-427e-a5a1-0396c925b1ba","Type":"ContainerStarted","Data":"fe398531d6472510553ae6a537e34e2973e6bef66edf7fd74c0c40db4ebb1626"}
Nov 02 14:57:08.443836 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.443800    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:57:08.461010 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.460983    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.514339 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.514310    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 02 14:57:08.516997 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.516963    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 02 14:57:08.534002 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.533975    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534156 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534010    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534156 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534030    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534156 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534048    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534156 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534111    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534340 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534161    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534340 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534273    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534340 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534313    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534486 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534430    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534542 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534486    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534542 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534522    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twppx\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534641 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534600    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.534727 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.534711    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.618313 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.618286    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 02 14:57:08.619286 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.619265    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:57:08.636043 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636015    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636058    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636078    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636102    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636230    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twppx\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636273    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636308    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636330    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636358    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636386    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636414    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636442    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636469    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.636532 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.636493    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.637207 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.637170    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.637523 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.637502    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.638720 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.638695    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.638873 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.638854    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.638987 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.638970    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.666389 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.666353    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 02 14:57:08.666748 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.666714    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 02 14:57:08.666748 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.666728    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 02 14:57:08.666748 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.666735    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 02 14:57:08.666985 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.666963    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 02 14:57:08.667067 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.667049    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-csl5z\""
Nov 02 14:57:08.667136 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.667099    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 02 14:57:08.669427 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.669405    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.669516 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.669432    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.678989 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.678961    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.679173 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.679141    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.679830 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.679809    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.680390 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.680351    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.697284 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.697213    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twppx\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\") pod \"alertmanager-main-0\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.795319 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.795288    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56964c66d-wjrwv"]
Nov 02 14:57:08.807380 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.807356    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.862088 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862044    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 02 14:57:08.862322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862125    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 02 14:57:08.862322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862147    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 02 14:57:08.862322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862154    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7bdbf3ril847s\""
Nov 02 14:57:08.862322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862241    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 02 14:57:08.862322 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862315    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5h9t5\""
Nov 02 14:57:08.862848 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.862821    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 02 14:57:08.863365 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.863344    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56964c66d-wjrwv"]
Nov 02 14:57:08.887351 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.887313    4470 generic.go:356] "Generic (PLEG): container finished" podID="6f1f75d6-4610-427e-a5a1-0396c925b1ba" containerID="4d568bcc2f0266d6adf33cea55de9f41e809854d9d5fff7f026c2ee83510feda" exitCode=0
Nov 02 14:57:08.887508 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.887358    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvwkz" event={"ID":"6f1f75d6-4610-427e-a5a1-0396c925b1ba","Type":"ContainerDied","Data":"4d568bcc2f0266d6adf33cea55de9f41e809854d9d5fff7f026c2ee83510feda"}
Nov 02 14:57:08.891514 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.891422    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:57:08.939949 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.939917    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940070 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.939967    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940070 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940049    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-grpc-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940140 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940088    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940140 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940115    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629f5487-7949-49df-8794-8bf26067bea3-metrics-client-ca\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940274 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940256    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940321 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940287    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940321 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940308    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629f5487-7949-49df-8794-8bf26067bea3-tmp\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:08.940397 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:08.940381    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x45r\" (UniqueName: \"kubernetes.io/projected/629f5487-7949-49df-8794-8bf26067bea3-kube-api-access-7x45r\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041153 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041120    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041314 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041165    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-grpc-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041314 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041226    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041314 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041252    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629f5487-7949-49df-8794-8bf26067bea3-metrics-client-ca\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041314 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041290    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041344    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041371    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629f5487-7949-49df-8794-8bf26067bea3-tmp\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.041634 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.041613    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x45r\" (UniqueName: \"kubernetes.io/projected/629f5487-7949-49df-8794-8bf26067bea3-kube-api-access-7x45r\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.042906 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.042872    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/629f5487-7949-49df-8794-8bf26067bea3-metrics-client-ca\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.043976 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.043946    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.044181 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.044152    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.044298 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.044285    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-grpc-tls\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.044470 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.044447    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.044574 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.044547    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629f5487-7949-49df-8794-8bf26067bea3-tmp\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.044632 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.044585    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.046688 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.046659    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.049357 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.049332    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/629f5487-7949-49df-8794-8bf26067bea3-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.105683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.105649    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x45r\" (UniqueName: \"kubernetes.io/projected/629f5487-7949-49df-8794-8bf26067bea3-kube-api-access-7x45r\") pod \"thanos-querier-56964c66d-wjrwv\" (UID: \"629f5487-7949-49df-8794-8bf26067bea3\") " pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.179313 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.179283    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:57:09.179467 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.179363    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:57:09.184316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.184282    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:57:09.187494 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.187475    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:09.364467 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.364435    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:57:09.367677 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:09.367648    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0563ab_62ee_47a7_9167_60867da949c5.slice/crio-e821ea3b9135a60a7981d02a48ea45e2a6144e752656d0a7df2191c4d5cb6dd4 WatchSource:0}: Error finding container e821ea3b9135a60a7981d02a48ea45e2a6144e752656d0a7df2191c4d5cb6dd4: Status 404 returned error can't find the container with id e821ea3b9135a60a7981d02a48ea45e2a6144e752656d0a7df2191c4d5cb6dd4
Nov 02 14:57:09.576169 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.576090    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56964c66d-wjrwv"]
Nov 02 14:57:09.579456 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:09.579418    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629f5487_7949_49df_8794_8bf26067bea3.slice/crio-221bd9505810543777f0210025d50690afa4ada3328331ac6cda8e10783f9375 WatchSource:0}: Error finding container 221bd9505810543777f0210025d50690afa4ada3328331ac6cda8e10783f9375: Status 404 returned error can't find the container with id 221bd9505810543777f0210025d50690afa4ada3328331ac6cda8e10783f9375
Nov 02 14:57:09.703811 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.703783    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:09.706830 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.706810    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.760376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.760337    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 02 14:57:09.765998 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.765964    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:09.856515 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856399    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856515 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856455    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856715 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856540    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856715 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856606    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856715 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856631    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8tw2\" (UniqueName: \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856715 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856656    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856866 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856722    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.856866 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.856789    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.893548 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.893510    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvwkz" event={"ID":"6f1f75d6-4610-427e-a5a1-0396c925b1ba","Type":"ContainerStarted","Data":"be98e6ef24028287bffbff135be4dffeb436acb84c6c187b29eea2b668285773"}
Nov 02 14:57:09.893732 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.893562    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bvwkz" event={"ID":"6f1f75d6-4610-427e-a5a1-0396c925b1ba","Type":"ContainerStarted","Data":"1df24ac0450d1726c38f7153a62872fd208efba27caf1a0c6a0d1dc391675e33"}
Nov 02 14:57:09.894909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.894883    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"221bd9505810543777f0210025d50690afa4ada3328331ac6cda8e10783f9375"}
Nov 02 14:57:09.896073 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.896049    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"e821ea3b9135a60a7981d02a48ea45e2a6144e752656d0a7df2191c4d5cb6dd4"}
Nov 02 14:57:09.900148 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.900127    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:57:09.957232 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957172    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8tw2\" (UniqueName: \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957247    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957273    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957304    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957348    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957463    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957526    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.957630 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957572    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.958309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.957970    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.958309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.958208    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.958309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.958182    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.958309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.958303    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.958703 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.958685    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.960020 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.959998    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:09.960020 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:09.960008    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:10.032617 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.032546    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bvwkz" podStartSLOduration=1.9533704360000002 podStartE2EDuration="4.032528188s" totalImagesPullingTime="2.006531295s" totalInitContainerRuntime="72.626457ms" isStatefulPod=true podCreationTimestamp="2026-11-02 14:57:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:10.031312302 +0000 UTC m=+36.469214443" watchObservedRunningTime="2026-11-02 14:57:10.032528188 +0000 UTC m=+36.470430331"
Nov 02 14:57:10.041932 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.041895    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8tw2\" (UniqueName: \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\") pod \"console-5dc9fdd6f-thqwh\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") " pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:10.317408 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.317370    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:10.692476 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.692442    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:10.708824 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:10.708784    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7e1b78_1d12_4cf5_9b25_b843d4c343c6.slice/crio-65df5aa9661b71611ef74845c848240088f7f12eaee4152c8fc50bf8583ec050 WatchSource:0}: Error finding container 65df5aa9661b71611ef74845c848240088f7f12eaee4152c8fc50bf8583ec050: Status 404 returned error can't find the container with id 65df5aa9661b71611ef74845c848240088f7f12eaee4152c8fc50bf8583ec050
Nov 02 14:57:10.839229 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.837824    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"]
Nov 02 14:57:10.840496 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.840356    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:10.901077 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.900757    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"]
Nov 02 14:57:10.904127 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.904089    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9fdd6f-thqwh" event={"ID":"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6","Type":"ContainerStarted","Data":"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"}
Nov 02 14:57:10.904293 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.904145    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9fdd6f-thqwh" event={"ID":"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6","Type":"ContainerStarted","Data":"65df5aa9661b71611ef74845c848240088f7f12eaee4152c8fc50bf8583ec050"}
Nov 02 14:57:10.918561 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.918528    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Nov 02 14:57:10.918561 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.918548    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-pcht2\""
Nov 02 14:57:10.966932 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.966786    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2ebe193b-b3c8-4197-b516-506c88b0dc33-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:10.967109 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:10.966930    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkt6g\" (UniqueName: \"kubernetes.io/projected/2ebe193b-b3c8-4197-b516-506c88b0dc33-kube-api-access-vkt6g\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.067402 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.067373    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-bc66565f-8htjh"]
Nov 02 14:57:11.070315 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.068783    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkt6g\" (UniqueName: \"kubernetes.io/projected/2ebe193b-b3c8-4197-b516-506c88b0dc33-kube-api-access-vkt6g\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.070315 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.069283    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2ebe193b-b3c8-4197-b516-506c88b0dc33-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.070315 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.069577    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.072516 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.072479    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2ebe193b-b3c8-4197-b516-506c88b0dc33-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.079669 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.079618    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5dc9fdd6f-thqwh" podStartSLOduration=2.079600344 podStartE2EDuration="2.079600344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:11.07857701 +0000 UTC m=+37.516479149" watchObservedRunningTime="2026-11-02 14:57:11.079600344 +0000 UTC m=+37.517502486"
Nov 02 14:57:11.122424 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122383    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 02 14:57:11.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122760    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 02 14:57:11.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122793    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1sgam3tr4mn33\""
Nov 02 14:57:11.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122883    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-jfqlw\""
Nov 02 14:57:11.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122915    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 02 14:57:11.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.122950    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 02 14:57:11.131652 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.131584    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkt6g\" (UniqueName: \"kubernetes.io/projected/2ebe193b-b3c8-4197-b516-506c88b0dc33-kube-api-access-vkt6g\") pod \"monitoring-plugin-77f4f76594-w4bkt\" (UID: \"2ebe193b-b3c8-4197-b516-506c88b0dc33\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.131817 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.131794    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bc66565f-8htjh"]
Nov 02 14:57:11.154667 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.154623    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:11.171905 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.171873    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172064 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.171937    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-metrics-server-audit-profiles\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172064 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.171980    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-client-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172312 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.172085    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8flk4\" (UniqueName: \"kubernetes.io/projected/6c2d4dc1-08a3-4e50-be9a-182def88ba72-kube-api-access-8flk4\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172377 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.172318    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-tls\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172428 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.172375    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-client-certs\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.172428 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.172413    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c2d4dc1-08a3-4e50-be9a-182def88ba72-audit-log\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273366 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273277    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-client-certs\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273366 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273325    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c2d4dc1-08a3-4e50-be9a-182def88ba72-audit-log\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273366 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273354    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273651 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273390    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-metrics-server-audit-profiles\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273651 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273417    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-client-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273651 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273445    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8flk4\" (UniqueName: \"kubernetes.io/projected/6c2d4dc1-08a3-4e50-be9a-182def88ba72-kube-api-access-8flk4\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273651 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273512    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-tls\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.273914 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.273799    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/6c2d4dc1-08a3-4e50-be9a-182def88ba72-audit-log\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.274620 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.274597    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-metrics-server-audit-profiles\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.275050 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.275024    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c2d4dc1-08a3-4e50-be9a-182def88ba72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.276140 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.276116    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-tls\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.276359 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.276343    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-secret-metrics-server-client-certs\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.277010 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.276983    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/6c2d4dc1-08a3-4e50-be9a-182def88ba72-client-ca-bundle\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.335359 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.335327    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8flk4\" (UniqueName: \"kubernetes.io/projected/6c2d4dc1-08a3-4e50-be9a-182def88ba72-kube-api-access-8flk4\") pod \"metrics-server-bc66565f-8htjh\" (UID: \"6c2d4dc1-08a3-4e50-be9a-182def88ba72\") " pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.391243 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.391179    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:11.533944 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.533817    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"]
Nov 02 14:57:11.537997 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:11.537966    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ebe193b_b3c8_4197_b516_506c88b0dc33.slice/crio-3d42fe129d3246f3dcef251891dea106e5cdbb9c10f892c37c705861b3a30e51 WatchSource:0}: Error finding container 3d42fe129d3246f3dcef251891dea106e5cdbb9c10f892c37c705861b3a30e51: Status 404 returned error can't find the container with id 3d42fe129d3246f3dcef251891dea106e5cdbb9c10f892c37c705861b3a30e51
Nov 02 14:57:11.664343 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.664305    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g5d86"
Nov 02 14:57:11.762906 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.762877    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-bc66565f-8htjh"]
Nov 02 14:57:11.766851 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:11.766816    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2d4dc1_08a3_4e50_be9a_182def88ba72.slice/crio-7e660439684ab777cecadbbd1c8aba4c06e0a870f1feba2552e6be916702fcce WatchSource:0}: Error finding container 7e660439684ab777cecadbbd1c8aba4c06e0a870f1feba2552e6be916702fcce: Status 404 returned error can't find the container with id 7e660439684ab777cecadbbd1c8aba4c06e0a870f1feba2552e6be916702fcce
Nov 02 14:57:11.909047 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.908961    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3" exitCode=0
Nov 02 14:57:11.909236 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.909057    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3"}
Nov 02 14:57:11.910610 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.910584    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt" event={"ID":"2ebe193b-b3c8-4197-b516-506c88b0dc33","Type":"ContainerStarted","Data":"3d42fe129d3246f3dcef251891dea106e5cdbb9c10f892c37c705861b3a30e51"}
Nov 02 14:57:11.912121 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:11.912082    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bc66565f-8htjh" event={"ID":"6c2d4dc1-08a3-4e50-be9a-182def88ba72","Type":"ContainerStarted","Data":"7e660439684ab777cecadbbd1c8aba4c06e0a870f1feba2552e6be916702fcce"}
Nov 02 14:57:12.919256 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:12.919218    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"3cfcc64ba02f7e4b00af5f3e2f9237831eb58ee820116f23104a5cd6460bfb69"}
Nov 02 14:57:12.919693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:12.919302    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"f77c1ace29bcd714a6238923ce6b910f30c5c77c9f61bff5d88dd1eaf7638634"}
Nov 02 14:57:12.919693 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:12.919333    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"cd8b084f87ee6bd6817a2aa7c8284ed598271199ee491cc8b834103a90dc383b"}
Nov 02 14:57:13.930017 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:13.929964    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt" event={"ID":"2ebe193b-b3c8-4197-b516-506c88b0dc33","Type":"ContainerStarted","Data":"d28cefa7e990b3cf6d9afd6197a6caf274ef45d8246c5fee6c688aa4c8d91f84"}
Nov 02 14:57:13.930450 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:13.930428    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:13.936141 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:13.936121    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt"
Nov 02 14:57:14.170082 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.170024    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-w4bkt" podStartSLOduration=2.306205346 podStartE2EDuration="4.17000395s" totalImagesPullingTime="1.863798604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:14.05334596 +0000 UTC m=+40.491248101" watchObservedRunningTime="2026-11-02 14:57:14.17000395 +0000 UTC m=+40.607906094"
Nov 02 14:57:14.938756 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.938719    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"e6515cfae394b69c1e84ddd500800b0723bcb1752ad0f2fa6afdb3ee962f44fc"}
Nov 02 14:57:14.939180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.938826    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"6452c850686a236a6ec8ed2ea4cdcba27549ea07368ecfc7ac12107fad764137"}
Nov 02 14:57:14.939180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.938863    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:14.939180 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.938898    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" event={"ID":"629f5487-7949-49df-8794-8bf26067bea3","Type":"ContainerStarted","Data":"25a6bc054cc5a51702499c79932529d91b37e3a7eb08608a6935def6129d8af0"}
Nov 02 14:57:14.940367 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.940338    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-bc66565f-8htjh" event={"ID":"6c2d4dc1-08a3-4e50-be9a-182def88ba72","Type":"ContainerStarted","Data":"094d875a5404007ca03ead20c0326be95b8cb7d4452fd09dd51b163f3f9ce76a"}
Nov 02 14:57:14.943574 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943549    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"}
Nov 02 14:57:14.943699 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943618    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"}
Nov 02 14:57:14.943699 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943667    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"}
Nov 02 14:57:14.943815 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943707    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"}
Nov 02 14:57:14.943815 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943745    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"}
Nov 02 14:57:14.943925 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:14.943885    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerStarted","Data":"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"}
Nov 02 14:57:15.062422 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:15.062329    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv" podStartSLOduration=2.945848275 podStartE2EDuration="7.062310707s" totalImagesPullingTime="4.116462432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:15.061544528 +0000 UTC m=+41.499446668" watchObservedRunningTime="2026-11-02 14:57:15.062310707 +0000 UTC m=+41.500212849"
Nov 02 14:57:15.313776 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:15.313685    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-bc66565f-8htjh" podStartSLOduration=2.211913556 podStartE2EDuration="4.313668768s" totalImagesPullingTime="2.101755212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:15.183986619 +0000 UTC m=+41.621888763" watchObservedRunningTime="2026-11-02 14:57:15.313668768 +0000 UTC m=+41.751570910"
Nov 02 14:57:15.314007 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:15.313988    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.443788209 podStartE2EDuration="7.313979706s" totalImagesPullingTime="3.851831134s" totalInitContainerRuntime="18.360363ms" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:15.313120093 +0000 UTC m=+41.751022236" watchObservedRunningTime="2026-11-02 14:57:15.313979706 +0000 UTC m=+41.751881847"
Nov 02 14:57:18.973958 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:18.973872    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:19.023361 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.023319    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:19.025756 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.025734    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.072735 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.072702    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:19.147284 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147254    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147298    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147384    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147412    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147457    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w28d\" (UniqueName: \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147672 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147556    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147672 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147595    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.147672 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.147621    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248576 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248488    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248576 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248529    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248576 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248569    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w28d\" (UniqueName: \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248594    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248629    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248667    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.248840 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248810    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249049 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.248868    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249491 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.249460    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249627 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.249510    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249697 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.249625    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249854 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.249829    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.249974 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.249957    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.251259 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.251235    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.251451 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.251432    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.309724 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.309693    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w28d\" (UniqueName: \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\") pod \"console-7c87f5d9d6-snqg5\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") " pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.335758 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.335720    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:19.712302 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.712265    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:19.715789 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:19.715764    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a94135_da48_42d3_a45c_be3936bcb6e9.slice/crio-878869614a3051e6ea3a9b9e8dc4781c11a38162a6e4f15cf258ac50cf986caa WatchSource:0}: Error finding container 878869614a3051e6ea3a9b9e8dc4781c11a38162a6e4f15cf258ac50cf986caa: Status 404 returned error can't find the container with id 878869614a3051e6ea3a9b9e8dc4781c11a38162a6e4f15cf258ac50cf986caa
Nov 02 14:57:19.967327 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.967234    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87f5d9d6-snqg5" event={"ID":"99a94135-da48-42d3-a45c-be3936bcb6e9","Type":"ContainerStarted","Data":"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"}
Nov 02 14:57:19.967327 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:19.967293    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87f5d9d6-snqg5" event={"ID":"99a94135-da48-42d3-a45c-be3936bcb6e9","Type":"ContainerStarted","Data":"878869614a3051e6ea3a9b9e8dc4781c11a38162a6e4f15cf258ac50cf986caa"}
Nov 02 14:57:20.088463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:20.088368    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7c87f5d9d6-snqg5" podStartSLOduration=2.088347673 podStartE2EDuration="2.088347673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:20.087600136 +0000 UTC m=+46.525502278" watchObservedRunningTime="2026-11-02 14:57:20.088347673 +0000 UTC m=+46.526249819"
Nov 02 14:57:20.318420 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:20.318328    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:20.955112 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:20.955085    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56964c66d-wjrwv"
Nov 02 14:57:25.173038 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.172998    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:25.222172 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.222138    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:57:25.248672 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.248640    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.288554 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.288523    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:57:25.315697 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315642    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.315889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315719    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcw7d\" (UniqueName: \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.315889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315782    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.315889 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315873    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.316054 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315910    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.316054 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315928    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.316054 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315948    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.316054 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.315995    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.416933 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.416887    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.416952    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.416984    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.416999    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417020    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417084    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417142 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417133    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417489 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417161    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcw7d\" (UniqueName: \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417599 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417561    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417860 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417830    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.417973 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417884    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.418044 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.417985    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.418207 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.418164    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.419514 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.419491    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.419577 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.419514    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.482657 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.482623    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcw7d\" (UniqueName: \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\") pod \"console-59ff6f468c-26m9f\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") " pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.571010 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.570971    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:25.947872 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.947831    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:57:25.951676 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:25.951642    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1795ca99_10ff_4dae_bde7_025acda0d94e.slice/crio-50316a301bede09b9bad2b360a8a2ce2b1ef2c80ba08f1e030c786f058058d7a WatchSource:0}: Error finding container 50316a301bede09b9bad2b360a8a2ce2b1ef2c80ba08f1e030c786f058058d7a: Status 404 returned error can't find the container with id 50316a301bede09b9bad2b360a8a2ce2b1ef2c80ba08f1e030c786f058058d7a
Nov 02 14:57:25.991839 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:25.991816    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff6f468c-26m9f" event={"ID":"1795ca99-10ff-4dae-bde7-025acda0d94e","Type":"ContainerStarted","Data":"50316a301bede09b9bad2b360a8a2ce2b1ef2c80ba08f1e030c786f058058d7a"}
Nov 02 14:57:26.997537 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:26.997501    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff6f468c-26m9f" event={"ID":"1795ca99-10ff-4dae-bde7-025acda0d94e","Type":"ContainerStarted","Data":"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"}
Nov 02 14:57:27.118564 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:27.118505    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59ff6f468c-26m9f" podStartSLOduration=2.11848913 podStartE2EDuration="2.11848913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:27.117962463 +0000 UTC m=+53.555864604" watchObservedRunningTime="2026-11-02 14:57:27.11848913 +0000 UTC m=+53.556391271"
Nov 02 14:57:29.335991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:29.335951    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:31.391513 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:31.391475    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:31.391878 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:31.391540    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:33.563684 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:57:33.563644    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 14:57:35.571824 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:35.571790    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:35.572259 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:35.571893    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:35.576793 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:35.576772    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:36.035563 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:36.035537    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:57:36.194315 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:36.194278    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:57:43.993743 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:43.993675    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5dc9fdd6f-thqwh" podUID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" containerName="console" containerID="cri-o://7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8" gracePeriod=15
Nov 02 14:57:44.281725 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.281637    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:44.394586 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394544    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394791 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394621    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394791 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394665    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394791 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394699    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394956 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394837    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394956 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394896    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.394956 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.394945    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.395111 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395011    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:57:44.395111 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395020    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\" (UniqueName: \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\") pod \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\" (UID: \"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6\") "
Nov 02 14:57:44.395242 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395115    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:44.395242 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395127    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:44.395344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395246    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:44.395463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395439    4470 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-oauth-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.395536 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395467    4470 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-tmp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.395536 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395478    4470 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.395536 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395486    4470 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.395536 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.395480    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:44.397213 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.397172    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "kube-api-access-p8tw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:57:44.397309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.397227    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:57:44.397309 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.397239    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config" pod "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" (UID: "ee7e1b78-1d12-4cf5-9b25-b843d4c343c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:57:44.495945 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.495900    4470 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-oauth-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.495945 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.495936    4470 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-console-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.495945 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.495947    4470 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-trusted-ca-bundle\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:44.495945 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:44.495956    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8tw2\" (UniqueName: \"kubernetes.io/projected/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6-kube-api-access-p8tw2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:45.065601 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.065566    4470 generic.go:356] "Generic (PLEG): container finished" podID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" containerID="7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8" exitCode=0
Nov 02 14:57:45.066036 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.065613    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dc9fdd6f-thqwh"
Nov 02 14:57:45.066036 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.065648    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9fdd6f-thqwh" event={"ID":"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6","Type":"ContainerDied","Data":"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"}
Nov 02 14:57:45.066036 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.065698    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dc9fdd6f-thqwh" event={"ID":"ee7e1b78-1d12-4cf5-9b25-b843d4c343c6","Type":"ContainerDied","Data":"65df5aa9661b71611ef74845c848240088f7f12eaee4152c8fc50bf8583ec050"}
Nov 02 14:57:45.066036 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.065718    4470 scope.go:122] "RemoveContainer" containerID="7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"
Nov 02 14:57:45.077378 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.077357    4470 scope.go:122] "RemoveContainer" containerID="7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"
Nov 02 14:57:45.077715 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:57:45.077689    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8\": container with ID starting with 7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8 not found: ID does not exist" containerID="7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"
Nov 02 14:57:45.077812 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.077723    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8"} err="failed to get container status \"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8\": rpc error: code = NotFound desc = could not find container \"7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8\": container with ID starting with 7d041ef6e75a455b0b83d9a2da3ae4a091e05029aee216d4b3de69dda09769f8 not found: ID does not exist"
Nov 02 14:57:45.229786 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.229751    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:45.279200 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.279163    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dc9fdd6f-thqwh"]
Nov 02 14:57:45.703718 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:45.703684    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" path="/var/lib/kubelet/pods/ee7e1b78-1d12-4cf5-9b25-b843d4c343c6/volumes"
Nov 02 14:57:50.193410 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.193346    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7c87f5d9d6-snqg5" podUID="99a94135-da48-42d3-a45c-be3936bcb6e9" containerName="console" containerID="cri-o://54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6" gracePeriod=15
Nov 02 14:57:50.438665 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.438642    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:50.546245 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546147    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546245 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546219    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546278    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546300    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546331    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546350    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546410    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546438    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\" (UniqueName: \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\") pod \"99a94135-da48-42d3-a45c-be3936bcb6e9\" (UID: \"99a94135-da48-42d3-a45c-be3936bcb6e9\") "
Nov 02 14:57:50.546760 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546520    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:57:50.546760 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546704    4470 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99a94135-da48-42d3-a45c-be3936bcb6e9-tmp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.546861 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546796    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:50.546861 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546806    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:50.546861 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.546815    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:50.547104 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.547085    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:57:50.548426 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.548398    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:57:50.548584 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.548564    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "kube-api-access-5w28d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:57:50.548647 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.548634    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config" pod "99a94135-da48-42d3-a45c-be3936bcb6e9" (UID: "99a94135-da48-42d3-a45c-be3936bcb6e9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:57:50.647973 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.647942    4470 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.647973 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.647970    4470 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.647973 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.647981    4470 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-trusted-ca-bundle\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.648234 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.647990    4470 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-console-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.648234 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.648001    4470 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99a94135-da48-42d3-a45c-be3936bcb6e9-oauth-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.648234 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.648009    4470 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99a94135-da48-42d3-a45c-be3936bcb6e9-console-oauth-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:50.648234 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:50.648018    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w28d\" (UniqueName: \"kubernetes.io/projected/99a94135-da48-42d3-a45c-be3936bcb6e9-kube-api-access-5w28d\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:57:51.090160 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.090122    4470 generic.go:356] "Generic (PLEG): container finished" podID="99a94135-da48-42d3-a45c-be3936bcb6e9" containerID="54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6" exitCode=0
Nov 02 14:57:51.090344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.090167    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c87f5d9d6-snqg5"
Nov 02 14:57:51.090344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.090224    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87f5d9d6-snqg5" event={"ID":"99a94135-da48-42d3-a45c-be3936bcb6e9","Type":"ContainerDied","Data":"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"}
Nov 02 14:57:51.090344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.090282    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c87f5d9d6-snqg5" event={"ID":"99a94135-da48-42d3-a45c-be3936bcb6e9","Type":"ContainerDied","Data":"878869614a3051e6ea3a9b9e8dc4781c11a38162a6e4f15cf258ac50cf986caa"}
Nov 02 14:57:51.090344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.090307    4470 scope.go:122] "RemoveContainer" containerID="54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"
Nov 02 14:57:51.098549 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.098531    4470 scope.go:122] "RemoveContainer" containerID="54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"
Nov 02 14:57:51.098848 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:57:51.098830    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6\": container with ID starting with 54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6 not found: ID does not exist" containerID="54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"
Nov 02 14:57:51.098897 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.098857    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6"} err="failed to get container status \"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6\": rpc error: code = NotFound desc = could not find container \"54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6\": container with ID starting with 54b69a39356bfc5f1cb0bd9fa49499de3eb863a771c842da932fc6dc813e2ce6 not found: ID does not exist"
Nov 02 14:57:51.253867 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.253831    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:51.303633 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.303602    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c87f5d9d6-snqg5"]
Nov 02 14:57:51.397204 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.397109    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:51.401239 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.401216    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-bc66565f-8htjh"
Nov 02 14:57:51.703373 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:51.703344    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99a94135-da48-42d3-a45c-be3936bcb6e9" path="/var/lib/kubelet/pods/99a94135-da48-42d3-a45c-be3936bcb6e9/volumes"
Nov 02 14:57:55.425740 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.425705    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d9fc999b6-686kf"]
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426007    4470 cpu_manager.go:414] "containerMap: removing container" podUID="99a94135-da48-42d3-a45c-be3936bcb6e9" containerName="console"
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426019    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99a94135-da48-42d3-a45c-be3936bcb6e9" containerName="console"
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426030    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" containerName="console"
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426035    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" containerName="console"
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426093    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="99a94135-da48-42d3-a45c-be3936bcb6e9" containerName="console"
Nov 02 14:57:55.426152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.426101    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee7e1b78-1d12-4cf5-9b25-b843d4c343c6" containerName="console"
Nov 02 14:57:55.428609 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.428593    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488634 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488598    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-oauth-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488634 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488635    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-oauth-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488862 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488663    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-service-ca\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488862 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488790    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c809cf8-9aaa-4cff-bc52-712a3de6f673-tmp\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488862 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488820    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dwld\" (UniqueName: \"kubernetes.io/projected/8c809cf8-9aaa-4cff-bc52-712a3de6f673-kube-api-access-5dwld\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.488862 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488849    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.489003 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488909    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.489003 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.488935    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-trusted-ca-bundle\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.533586 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.533546    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9fc999b6-686kf"]
Nov 02 14:57:55.589856 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.589809    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.589856 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.589863    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-trusted-ca-bundle\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590122 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590034    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-oauth-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590122 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590075    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-oauth-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590122 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590103    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-service-ca\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590317 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590139    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c809cf8-9aaa-4cff-bc52-712a3de6f673-tmp\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590317 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590160    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dwld\" (UniqueName: \"kubernetes.io/projected/8c809cf8-9aaa-4cff-bc52-712a3de6f673-kube-api-access-5dwld\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590317 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590221    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590801 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590770    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-oauth-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590946 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590798    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c809cf8-9aaa-4cff-bc52-712a3de6f673-tmp\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.590946 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.590803    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-trusted-ca-bundle\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.591070 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.591048    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-service-ca\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.591116 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.591089    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.592489 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.592471    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-serving-cert\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.592691 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.592671    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8c809cf8-9aaa-4cff-bc52-712a3de6f673-console-oauth-config\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.651774 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.651738    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dwld\" (UniqueName: \"kubernetes.io/projected/8c809cf8-9aaa-4cff-bc52-712a3de6f673-kube-api-access-5dwld\") pod \"console-5d9fc999b6-686kf\" (UID: \"8c809cf8-9aaa-4cff-bc52-712a3de6f673\") " pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:55.738983 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:55.738940    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:57:56.114917 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:56.114852    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9fc999b6-686kf"]
Nov 02 14:57:56.117493 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:57:56.117460    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c809cf8_9aaa_4cff_bc52_712a3de6f673.slice/crio-d0c7f9c8327db289107038793a078bf552ee64530771c26679c8989c7d7c633d WatchSource:0}: Error finding container d0c7f9c8327db289107038793a078bf552ee64530771c26679c8989c7d7c633d: Status 404 returned error can't find the container with id d0c7f9c8327db289107038793a078bf552ee64530771c26679c8989c7d7c633d
Nov 02 14:57:57.121209 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:57.121155    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9fc999b6-686kf" event={"ID":"8c809cf8-9aaa-4cff-bc52-712a3de6f673","Type":"ContainerStarted","Data":"b556de8484683862df9fce66a7714a0ea5b9e0b3e542c5c3bad037334feebfc2"}
Nov 02 14:57:57.121620 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:57.121236    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9fc999b6-686kf" event={"ID":"8c809cf8-9aaa-4cff-bc52-712a3de6f673","Type":"ContainerStarted","Data":"d0c7f9c8327db289107038793a078bf552ee64530771c26679c8989c7d7c633d"}
Nov 02 14:57:57.289805 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:57:57.289742    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5d9fc999b6-686kf" podStartSLOduration=2.289725437 podStartE2EDuration="2.289725437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:57:57.239755254 +0000 UTC m=+83.677657394" watchObservedRunningTime="2026-11-02 14:57:57.289725437 +0000 UTC m=+83.727627572"
Nov 02 14:58:01.213702 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.213642    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59dcbcc7bd-h8grb" podUID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" containerName="console" containerID="cri-o://da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680" gracePeriod=15
Nov 02 14:58:01.507490 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.507459    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632486    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632548    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632601    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\" (UniqueName: \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632644    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632696    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632754    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\" (UniqueName: \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633220 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.632791    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\") pod \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\" (UID: \"60adcf0b-20e3-4abe-ba70-0d05ff446a11\") "
Nov 02 14:58:01.633820 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.633564    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:01.633945 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.633917    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:01.634616 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.634590    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:01.635379 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.635337    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:58:01.637169 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.637141    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:01.637169 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.637157    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "kube-api-access-lxprr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:58:01.637881 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.637857    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert" pod "60adcf0b-20e3-4abe-ba70-0d05ff446a11" (UID: "60adcf0b-20e3-4abe-ba70-0d05ff446a11"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:01.736733 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.736693    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxprr\" (UniqueName: \"kubernetes.io/projected/60adcf0b-20e3-4abe-ba70-0d05ff446a11-kube-api-access-lxprr\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.736983 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.736966    4470 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-oauth-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.737092 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.737078    4470 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.737178 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.737167    4470 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60adcf0b-20e3-4abe-ba70-0d05ff446a11-tmp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.737303 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.737285    4470 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.737401 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.737385    4470 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:01.737488 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:01.737474    4470 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/60adcf0b-20e3-4abe-ba70-0d05ff446a11-console-oauth-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:02.140881 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.140848    4470 generic.go:356] "Generic (PLEG): container finished" podID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" containerID="da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680" exitCode=0
Nov 02 14:58:02.141097 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.140905    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcbcc7bd-h8grb"
Nov 02 14:58:02.141097 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.140940    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcbcc7bd-h8grb" event={"ID":"60adcf0b-20e3-4abe-ba70-0d05ff446a11","Type":"ContainerDied","Data":"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"}
Nov 02 14:58:02.141097 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.140996    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcbcc7bd-h8grb" event={"ID":"60adcf0b-20e3-4abe-ba70-0d05ff446a11","Type":"ContainerDied","Data":"a756b190158d2d151599d3c22652a53b6233b78659aff48eacf8ad55926a55b6"}
Nov 02 14:58:02.141097 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.141017    4470 scope.go:122] "RemoveContainer" containerID="da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"
Nov 02 14:58:02.149390 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.149367    4470 scope.go:122] "RemoveContainer" containerID="da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"
Nov 02 14:58:02.149738 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:02.149714    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680\": container with ID starting with da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680 not found: ID does not exist" containerID="da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"
Nov 02 14:58:02.149846 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.149745    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680"} err="failed to get container status \"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680\": rpc error: code = NotFound desc = could not find container \"da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680\": container with ID starting with da45773362b09c6afa0f73f54bad205c4e2ea673608c7a32cceb6a58a72b1680 not found: ID does not exist"
Nov 02 14:58:02.304885 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.304851    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:58:02.354707 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.354672    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59dcbcc7bd-h8grb"]
Nov 02 14:58:02.456474 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456435    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:02.456801 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456774    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="alertmanager" containerID="cri-o://139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c" gracePeriod=120
Nov 02 14:58:02.456894 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456841    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-metric" containerID="cri-o://9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd" gracePeriod=120
Nov 02 14:58:02.456950 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456858    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="prom-label-proxy" containerID="cri-o://94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b" gracePeriod=120
Nov 02 14:58:02.456950 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456898    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy" containerID="cri-o://b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684" gracePeriod=120
Nov 02 14:58:02.457040 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456918    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="config-reloader" containerID="cri-o://5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea" gracePeriod=120
Nov 02 14:58:02.457040 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:02.456880    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-web" containerID="cri-o://3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984" gracePeriod=120
Nov 02 14:58:03.150241 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150179    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b" exitCode=0
Nov 02 14:58:03.150241 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150224    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd" exitCode=0
Nov 02 14:58:03.150241 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150230    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684" exitCode=0
Nov 02 14:58:03.150241 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150238    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea" exitCode=0
Nov 02 14:58:03.150241 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150247    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c" exitCode=0
Nov 02 14:58:03.150656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150261    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"}
Nov 02 14:58:03.150656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150371    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"}
Nov 02 14:58:03.150656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150420    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"}
Nov 02 14:58:03.150656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150458    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"}
Nov 02 14:58:03.150656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.150496    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"}
Nov 02 14:58:03.564439 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:03.564385    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 14:58:03.694776 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.694751    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:03.702967 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.702945    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" path="/var/lib/kubelet/pods/60adcf0b-20e3-4abe-ba70-0d05ff446a11/volumes"
Nov 02 14:58:03.858316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858156    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858212    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858244    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858272    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858316 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858310    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858445    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858501    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858547    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858581    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858613    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858653    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:03.858695 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858658    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.859080 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858765    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.859080 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.858806    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\") pod \"dc0563ab-62ee-47a7-9167-60867da949c5\" (UID: \"dc0563ab-62ee-47a7-9167-60867da949c5\") "
Nov 02 14:58:03.859221 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.859087    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-metrics-client-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.859514 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.859479    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:58:03.860182 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.860152    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:03.861152 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.861112    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:58:03.861449 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.861417    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.861540 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.861521    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "kube-api-access-twppx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:58:03.861670 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.861638    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.861848 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.861823    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.862040 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.862017    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.862453 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.862437    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:58:03.863728 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.863712    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.866408 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.866392    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.873559 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.873532    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config" pod "dc0563ab-62ee-47a7-9167-60867da949c5" (UID: "dc0563ab-62ee-47a7-9167-60867da949c5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:03.959909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959879    4470 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-main-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.959909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959906    4470 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.959909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959918    4470 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-tls-assets\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959928    4470 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959938    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twppx\" (UniqueName: \"kubernetes.io/projected/dc0563ab-62ee-47a7-9167-60867da949c5-kube-api-access-twppx\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959947    4470 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-cluster-tls-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959957    4470 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-main-db\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959965    4470 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959974    4470 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959982    4470 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc0563ab-62ee-47a7-9167-60867da949c5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959991    4470 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc0563ab-62ee-47a7-9167-60867da949c5-config-out\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:03.960147 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:03.959999    4470 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc0563ab-62ee-47a7-9167-60867da949c5-web-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:04.157930 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.157851    4470 generic.go:356] "Generic (PLEG): container finished" podID="dc0563ab-62ee-47a7-9167-60867da949c5" containerID="3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984" exitCode=0
Nov 02 14:58:04.157930 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.157896    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"}
Nov 02 14:58:04.158110 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.157944    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.158110 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.157955    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"dc0563ab-62ee-47a7-9167-60867da949c5","Type":"ContainerDied","Data":"e821ea3b9135a60a7981d02a48ea45e2a6144e752656d0a7df2191c4d5cb6dd4"}
Nov 02 14:58:04.158110 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.157986    4470 scope.go:122] "RemoveContainer" containerID="94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"
Nov 02 14:58:04.165988 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.165961    4470 scope.go:122] "RemoveContainer" containerID="9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"
Nov 02 14:58:04.173759 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.173738    4470 scope.go:122] "RemoveContainer" containerID="b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"
Nov 02 14:58:04.180837 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.180817    4470 scope.go:122] "RemoveContainer" containerID="3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"
Nov 02 14:58:04.187553 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.187530    4470 scope.go:122] "RemoveContainer" containerID="5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"
Nov 02 14:58:04.194587 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.194567    4470 scope.go:122] "RemoveContainer" containerID="139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"
Nov 02 14:58:04.203656 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.203636    4470 scope.go:122] "RemoveContainer" containerID="ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3"
Nov 02 14:58:04.210907 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.210887    4470 scope.go:122] "RemoveContainer" containerID="94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"
Nov 02 14:58:04.211223 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.211197    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b\": container with ID starting with 94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b not found: ID does not exist" containerID="94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"
Nov 02 14:58:04.211353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211234    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b"} err="failed to get container status \"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b\": rpc error: code = NotFound desc = could not find container \"94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b\": container with ID starting with 94c3f7cfc0acd1f5a198b71427c94edcf518fd02117281a78148747e2abf547b not found: ID does not exist"
Nov 02 14:58:04.211353 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211260    4470 scope.go:122] "RemoveContainer" containerID="9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"
Nov 02 14:58:04.211552 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.211529    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd\": container with ID starting with 9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd not found: ID does not exist" containerID="9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"
Nov 02 14:58:04.211594 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211560    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd"} err="failed to get container status \"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd\": rpc error: code = NotFound desc = could not find container \"9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd\": container with ID starting with 9ec648a8e288b266ca22d6b7473b190f1314ddec9dc718e36da46f0760a27dcd not found: ID does not exist"
Nov 02 14:58:04.211594 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211582    4470 scope.go:122] "RemoveContainer" containerID="b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"
Nov 02 14:58:04.211841 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.211812    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684\": container with ID starting with b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684 not found: ID does not exist" containerID="b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"
Nov 02 14:58:04.211884 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211847    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684"} err="failed to get container status \"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684\": rpc error: code = NotFound desc = could not find container \"b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684\": container with ID starting with b5510e6cf1902c455dd366ee6863c2e08dec58d7abdf23d0c8afcf42f6c56684 not found: ID does not exist"
Nov 02 14:58:04.211884 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.211863    4470 scope.go:122] "RemoveContainer" containerID="3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"
Nov 02 14:58:04.212139 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.212120    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984\": container with ID starting with 3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984 not found: ID does not exist" containerID="3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"
Nov 02 14:58:04.212210 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212144    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984"} err="failed to get container status \"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984\": rpc error: code = NotFound desc = could not find container \"3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984\": container with ID starting with 3f5cf99d1221697fa9fa3234da1c4e5f9211182182a7291f3cc62ea36dbd6984 not found: ID does not exist"
Nov 02 14:58:04.212210 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212161    4470 scope.go:122] "RemoveContainer" containerID="5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"
Nov 02 14:58:04.212419 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.212401    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea\": container with ID starting with 5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea not found: ID does not exist" containerID="5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"
Nov 02 14:58:04.212456 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212424    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea"} err="failed to get container status \"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea\": rpc error: code = NotFound desc = could not find container \"5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea\": container with ID starting with 5b7bc282e65fbc5df5777922e643165e4400445eb087cbcffa4c9a6459c549ea not found: ID does not exist"
Nov 02 14:58:04.212456 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212438    4470 scope.go:122] "RemoveContainer" containerID="139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"
Nov 02 14:58:04.212672 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.212652    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c\": container with ID starting with 139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c not found: ID does not exist" containerID="139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"
Nov 02 14:58:04.212711 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212677    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c"} err="failed to get container status \"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c\": rpc error: code = NotFound desc = could not find container \"139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c\": container with ID starting with 139bbd2e9a5b676200d62c413975e6397472007b7a55eaf3ecaed465f70d009c not found: ID does not exist"
Nov 02 14:58:04.212711 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212691    4470 scope.go:122] "RemoveContainer" containerID="ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3"
Nov 02 14:58:04.212908 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:04.212891    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3\": container with ID starting with ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3 not found: ID does not exist" containerID="ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3"
Nov 02 14:58:04.212952 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.212912    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3"} err="failed to get container status \"ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3\": rpc error: code = NotFound desc = could not find container \"ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3\": container with ID starting with ce2e28cbdb2f93e59bcb63908062f2d8a96539250e89eb7ecdd62295c86b1ec3 not found: ID does not exist"
Nov 02 14:58:04.325805 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.325770    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:04.330301 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.330278    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:04.367069 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367042    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:04.367332 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367320    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-web"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367333    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-web"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367349    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="init-config-reloader"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367354    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="init-config-reloader"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367360    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="prom-label-proxy"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367365    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="prom-label-proxy"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367374    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="alertmanager"
Nov 02 14:58:04.367376 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367379    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="alertmanager"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367387    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="config-reloader"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367391    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="config-reloader"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367397    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367402    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367408    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-metric"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367413    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-metric"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367421    4470 cpu_manager.go:414] "containerMap: removing container" podUID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" containerName="console"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367426    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" containerName="console"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367469    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="prom-label-proxy"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367477    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="alertmanager"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367484    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="config-reloader"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367490    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367495    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-metric"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367501    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="60adcf0b-20e3-4abe-ba70-0d05ff446a11" containerName="console"
Nov 02 14:58:04.367573 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.367508    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" containerName="kube-rbac-proxy-web"
Nov 02 14:58:04.370995 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.370978    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.423814 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423707    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 02 14:58:04.423814 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423731    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 02 14:58:04.424085 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423926    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-csl5z\""
Nov 02 14:58:04.424085 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423934    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 02 14:58:04.424085 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423972    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 02 14:58:04.424085 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.423979    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 02 14:58:04.424085 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.424036    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 02 14:58:04.424773 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.424684    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 02 14:58:04.424773 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.424741    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 02 14:58:04.425029 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.425005    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 02 14:58:04.464370 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464330    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464370 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464369    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464589 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464445    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464589 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464516    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464589 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464541    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfwqh\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-kube-api-access-hfwqh\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464603    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-out\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464621    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464683 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464666    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464770 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464703    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464770 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464741    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464833 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464769    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-web-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464833 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464797    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.464833 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.464820    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566032 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.565985    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566032 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566034    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566062    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566182    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566252    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566281    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfwqh\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-kube-api-access-hfwqh\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566316    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-out\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566337    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566378    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566436    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566496    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566543    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-web-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566582    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.566898 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.566666    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.567511 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.567361    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.568590 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.568560    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2a70169-94fd-4dc9-b3e2-6483373fe14c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.569248 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569183    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-volume\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.569390 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569360    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.569492 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569449    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.569824 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569793    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.569989 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569967    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.570076 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.569998    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.570429 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.570410    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c2a70169-94fd-4dc9-b3e2-6483373fe14c-config-out\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.570838 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.570822    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-web-config\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.571259 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.571242    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c2a70169-94fd-4dc9-b3e2-6483373fe14c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.590219 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.589622    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:04.629372 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.629342    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfwqh\" (UniqueName: \"kubernetes.io/projected/c2a70169-94fd-4dc9-b3e2-6483373fe14c-kube-api-access-hfwqh\") pod \"alertmanager-main-0\" (UID: \"c2a70169-94fd-4dc9-b3e2-6483373fe14c\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:04.688114 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:04.688083    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 02 14:58:05.120982 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.120911    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 02 14:58:05.124685 ip-10-0-139-197 kubenswrapper[4470]: W1102 14:58:05.124660    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a70169_94fd_4dc9_b3e2_6483373fe14c.slice/crio-6a808f0dcb4287476ee1515f2ab3f1ec75780b8d04909dc2f5720d982a189651 WatchSource:0}: Error finding container 6a808f0dcb4287476ee1515f2ab3f1ec75780b8d04909dc2f5720d982a189651: Status 404 returned error can't find the container with id 6a808f0dcb4287476ee1515f2ab3f1ec75780b8d04909dc2f5720d982a189651
Nov 02 14:58:05.162795 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.162756    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"6a808f0dcb4287476ee1515f2ab3f1ec75780b8d04909dc2f5720d982a189651"}
Nov 02 14:58:05.703989 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.703943    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc0563ab-62ee-47a7-9167-60867da949c5" path="/var/lib/kubelet/pods/dc0563ab-62ee-47a7-9167-60867da949c5/volumes"
Nov 02 14:58:05.739394 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.739357    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:58:05.739537 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.739448    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:58:05.744148 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:05.744126    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:58:06.169088 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:06.169007    4470 generic.go:356] "Generic (PLEG): container finished" podID="c2a70169-94fd-4dc9-b3e2-6483373fe14c" containerID="7a4dcb8d7cb710a617fedfd5186833a6b633541c69292a5f17e6290afde714c2" exitCode=0
Nov 02 14:58:06.169248 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:06.169098    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerDied","Data":"7a4dcb8d7cb710a617fedfd5186833a6b633541c69292a5f17e6290afde714c2"}
Nov 02 14:58:06.173313 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:06.173285    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d9fc999b6-686kf"
Nov 02 14:58:06.451149 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:06.451119    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:58:07.176991 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.176950    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"013e64aa2c4aa403e00643f369f5b6683127ed8ac091f4c11b50783c50351d16"}
Nov 02 14:58:07.177445 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.177024    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"c01b69997066204a1c92adeb00c26db795df27f5d1d8bfe2089f921fdf88cca1"}
Nov 02 14:58:07.177445 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.177069    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"1dcf0aedfa947ea87252d819afc855810f04dad319d5e2e0f455c729d2a5049e"}
Nov 02 14:58:07.177445 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.177098    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"528031cf01db88846a672b51a27f29d316ad6326481b0b274cf53399c91cb3c6"}
Nov 02 14:58:07.177445 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.177129    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"91d1953d2b0c2d0207b8312422d2d9940af41f65df0645734f4c470cbaabcbc1"}
Nov 02 14:58:07.177445 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.177153    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c2a70169-94fd-4dc9-b3e2-6483373fe14c","Type":"ContainerStarted","Data":"df71c2b23982e4f69e603b7f17f26662594fbcdb59d372af7c4fa9169be14482"}
Nov 02 14:58:07.307800 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:07.307745    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.29600839 podStartE2EDuration="3.307726593s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.718203ms" isStatefulPod=false podCreationTimestamp="2026-11-02 14:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:58:07.306750067 +0000 UTC m=+93.744652211" watchObservedRunningTime="2026-11-02 14:58:07.307726593 +0000 UTC m=+93.745628735"
Nov 02 14:58:31.475165 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.475120    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59ff6f468c-26m9f" podUID="1795ca99-10ff-4dae-bde7-025acda0d94e" containerName="console" containerID="cri-o://d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546" gracePeriod=15
Nov 02 14:58:31.713533 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.713508    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:58:31.784246 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784153    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784246 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784236    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\" (UniqueName: \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784263    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784317    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784369    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784408    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784463 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784444    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784727 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784507    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\") pod \"1795ca99-10ff-4dae-bde7-025acda0d94e\" (UID: \"1795ca99-10ff-4dae-bde7-025acda0d94e\") "
Nov 02 14:58:31.784909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784855    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:31.784909 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784870    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:31.785074 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784935    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 14:58:31.785074 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.784985    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:31.785074 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.785049    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 14:58:31.786571 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.786551    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:31.786645 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.786582    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 14:58:31.786684 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.786669    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d" pod "1795ca99-10ff-4dae-bde7-025acda0d94e" (UID: "1795ca99-10ff-4dae-bde7-025acda0d94e"). InnerVolumeSpecName "kube-api-access-dcw7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 14:58:31.885815 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885775    4470 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-trusted-ca-bundle\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.885815 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885806    4470 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.885815 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885818    4470 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1795ca99-10ff-4dae-bde7-025acda0d94e-tmp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.886068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885828    4470 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-console-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.886068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885837    4470 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1795ca99-10ff-4dae-bde7-025acda0d94e-oauth-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.886068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885848    4470 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-serving-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.886068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885858    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcw7d\" (UniqueName: \"kubernetes.io/projected/1795ca99-10ff-4dae-bde7-025acda0d94e-kube-api-access-dcw7d\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:31.886068 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:31.885867    4470 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1795ca99-10ff-4dae-bde7-025acda0d94e-console-oauth-config\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 14:58:32.276344 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.276308    4470 generic.go:356] "Generic (PLEG): container finished" podID="1795ca99-10ff-4dae-bde7-025acda0d94e" containerID="d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546" exitCode=0
Nov 02 14:58:32.276531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.276365    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff6f468c-26m9f"
Nov 02 14:58:32.276531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.276398    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff6f468c-26m9f" event={"ID":"1795ca99-10ff-4dae-bde7-025acda0d94e","Type":"ContainerDied","Data":"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"}
Nov 02 14:58:32.276531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.276453    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff6f468c-26m9f" event={"ID":"1795ca99-10ff-4dae-bde7-025acda0d94e","Type":"ContainerDied","Data":"50316a301bede09b9bad2b360a8a2ce2b1ef2c80ba08f1e030c786f058058d7a"}
Nov 02 14:58:32.276531 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.276484    4470 scope.go:122] "RemoveContainer" containerID="d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"
Nov 02 14:58:32.285777 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.285740    4470 scope.go:122] "RemoveContainer" containerID="d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"
Nov 02 14:58:32.286116 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:32.286092    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546\": container with ID starting with d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546 not found: ID does not exist" containerID="d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"
Nov 02 14:58:32.286233 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.286123    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546"} err="failed to get container status \"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546\": rpc error: code = NotFound desc = could not find container \"d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546\": container with ID starting with d22d366938db01ff568a7692f5a3e4a75d59912b48bfd7a78ee3b2a463a65546 not found: ID does not exist"
Nov 02 14:58:32.441076 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.441035    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:58:32.490526 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:32.490486    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59ff6f468c-26m9f"]
Nov 02 14:58:33.563935 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:58:33.563855    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 14:58:33.702674 ip-10-0-139-197 kubenswrapper[4470]: I1102 14:58:33.702650    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1795ca99-10ff-4dae-bde7-025acda0d94e" path="/var/lib/kubelet/pods/1795ca99-10ff-4dae-bde7-025acda0d94e/volumes"
Nov 02 14:59:03.564248 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:59:03.564208    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 14:59:33.564255 ip-10-0-139-197 kubenswrapper[4470]: E1102 14:59:33.564213    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:00:03.563125 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:00:03.563084    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:00:08.268355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.268315    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:00:08.268788 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.268618    4470 cpu_manager.go:414] "containerMap: removing container" podUID="1795ca99-10ff-4dae-bde7-025acda0d94e" containerName="console"
Nov 02 15:00:08.268788 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.268628    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1795ca99-10ff-4dae-bde7-025acda0d94e" containerName="console"
Nov 02 15:00:08.268788 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.268681    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="1795ca99-10ff-4dae-bde7-025acda0d94e" containerName="console"
Nov 02 15:00:08.271590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.271571    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.329142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.329106    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 15:00:08.348966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.348936    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.348991    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.349107    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.349234    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.349269    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkfz\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.349344    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.349402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.349376    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.373955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.373889    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 15:00:08.373955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.373908    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2kqpp\""
Nov 02 15:00:08.373955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.373894    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 15:00:08.373955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.373951    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 15:00:08.374359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.374069    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 15:00:08.374359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.374120    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 15:00:08.375432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.375409    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:00:08.450092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450046    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450096    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450135    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450153    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkfz\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450178    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450212    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450236    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.450918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.450890    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.452824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.452795    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.452926 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.452879    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.452987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.452968    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.453225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.453178    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.511396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.511363    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.511587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.511566    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkfz\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\") pod \"istiod-openshift-gateway-7876bfcb74-klqrq\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:08.583557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:08.583471    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:09.013142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:09.013106    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:00:09.016121 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:00:09.016095    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdc343b_6af9_48da_a0a4_654450468e6b.slice/crio-faed95c92de3a1b8f1084dbac58a06b917de2b2ee5f38c59bbf019cda5d18c23 WatchSource:0}: Error finding container faed95c92de3a1b8f1084dbac58a06b917de2b2ee5f38c59bbf019cda5d18c23: Status 404 returned error can't find the container with id faed95c92de3a1b8f1084dbac58a06b917de2b2ee5f38c59bbf019cda5d18c23
Nov 02 15:00:09.620466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:09.620432    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" event={"ID":"5fdc343b-6af9-48da-a0a4-654450468e6b","Type":"ContainerStarted","Data":"faed95c92de3a1b8f1084dbac58a06b917de2b2ee5f38c59bbf019cda5d18c23"}
Nov 02 15:00:13.068633 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:13.068601    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:00:13.068896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:13.068665    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:00:13.637584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:13.637542    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" event={"ID":"5fdc343b-6af9-48da-a0a4-654450468e6b","Type":"ContainerStarted","Data":"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"}
Nov 02 15:00:13.637808 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:13.637655    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:13.808805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:13.808738    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" podStartSLOduration=1.758935066 podStartE2EDuration="5.808721495s" totalImagesPullingTime="4.049786429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:00:13.807788526 +0000 UTC m=+220.245690667" watchObservedRunningTime="2026-11-02 15:00:13.808721495 +0000 UTC m=+220.246623636"
Nov 02 15:00:14.642661 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:14.642626    4470 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-klqrq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:00:14.643066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:14.642681    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:00:17.642873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:00:17.642838    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:00:33.563882 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:00:33.563844    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:01:03.540237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.540184    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"]
Nov 02 15:01:03.543496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.543474    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.565987 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:01:03.565942    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:01:03.607933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.607898    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.608138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.607958    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-manifests\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.608138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.608021    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.608138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.608050    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-tmp\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.608138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.608070    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r688\" (UniqueName: \"kubernetes.io/projected/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-kube-api-access-9r688\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.608345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.608213    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.654241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.654197    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"]
Nov 02 15:01:03.668558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.668520    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 15:01:03.668756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.668734    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 15:01:03.668817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.668752    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 15:01:03.668817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.668745    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-5tklz\""
Nov 02 15:01:03.669687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.669673    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 15:01:03.669743 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.669686    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 15:01:03.709081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709043    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-manifests\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709080    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709110    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-tmp\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709128    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r688\" (UniqueName: \"kubernetes.io/projected/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-kube-api-access-9r688\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709147    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709182    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.709488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.709463    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-manifests\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.711666 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.711643    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-tmp\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.711877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.711855    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-webhook-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.711921 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.711855    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.712588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.712564    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-metrics-cert\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.776428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.776392    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r688\" (UniqueName: \"kubernetes.io/projected/bdd42662-54eb-41b4-ac0a-0894dd92fd4a-kube-api-access-9r688\") pod \"opendatahub-operator-controller-manager-55fb46c94b-twpzx\" (UID: \"bdd42662-54eb-41b4-ac0a-0894dd92fd4a\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:03.859060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:03.858957    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:04.433974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:04.433873    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"]
Nov 02 15:01:04.436931 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:01:04.436892    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdd42662_54eb_41b4_ac0a_0894dd92fd4a.slice/crio-151428a56fc3c41170bfd71a0050036daf3be58861107958f4acd6950b38ed98 WatchSource:0}: Error finding container 151428a56fc3c41170bfd71a0050036daf3be58861107958f4acd6950b38ed98: Status 404 returned error can't find the container with id 151428a56fc3c41170bfd71a0050036daf3be58861107958f4acd6950b38ed98
Nov 02 15:01:04.829620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:04.829529    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx" event={"ID":"bdd42662-54eb-41b4-ac0a-0894dd92fd4a","Type":"ContainerStarted","Data":"151428a56fc3c41170bfd71a0050036daf3be58861107958f4acd6950b38ed98"}
Nov 02 15:01:08.848553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:08.848508    4470 generic.go:356] "Generic (PLEG): container finished" podID="bdd42662-54eb-41b4-ac0a-0894dd92fd4a" containerID="8905f2162b921a4db5787d026efa7b18b48870cea2d10b537013716ea1f045a3" exitCode=0
Nov 02 15:01:08.848953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:08.848575    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx" event={"ID":"bdd42662-54eb-41b4-ac0a-0894dd92fd4a","Type":"ContainerDied","Data":"8905f2162b921a4db5787d026efa7b18b48870cea2d10b537013716ea1f045a3"}
Nov 02 15:01:09.853911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:09.853874    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx" event={"ID":"bdd42662-54eb-41b4-ac0a-0894dd92fd4a","Type":"ContainerStarted","Data":"cbf78687f2c9ec501b8161e67e4b58923a41bf6bb44e569b7f4a5e48de42d48a"}
Nov 02 15:01:09.854376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:09.854016    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:10.000640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:09.999221    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx" podStartSLOduration=2.4083850079999998 podStartE2EDuration="6.9991728s" totalImagesPullingTime="4.181427703s" totalInitContainerRuntime="409.360089ms" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:01:09.99429082 +0000 UTC m=+276.432192965" watchObservedRunningTime="2026-11-02 15:01:09.9991728 +0000 UTC m=+276.437074941"
Nov 02 15:01:20.860633 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:20.860598    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-55fb46c94b-twpzx"
Nov 02 15:01:33.564596 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:01:33.564551    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:01:33.713949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:33.713922    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:01:33.714154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:33.713962    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:01:34.702348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.702309    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-6m9kg"]
Nov 02 15:01:34.708786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.708756    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.743200 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.743163    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-bound-sa-token\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.743363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.743336    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f585x\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-kube-api-access-f585x\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.813204 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.813164    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 02 15:01:34.813420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.813179    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-bjvl8\""
Nov 02 15:01:34.813420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.813344    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 02 15:01:34.814669 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.814641    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-6m9kg"]
Nov 02 15:01:34.844288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.844246    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f585x\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-kube-api-access-f585x\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.844486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.844309    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-bound-sa-token\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.905342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.905304    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-bound-sa-token\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:34.905635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:34.905497    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f585x\" (UniqueName: \"kubernetes.io/projected/3d093745-9f02-40b2-a5fe-3eef57e1fb6a-kube-api-access-f585x\") pod \"cert-manager-597b95cb84-6m9kg\" (UID: \"3d093745-9f02-40b2-a5fe-3eef57e1fb6a\") " pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:35.030231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:35.030115    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-6m9kg"
Nov 02 15:01:35.400166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:35.400090    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-6m9kg"]
Nov 02 15:01:35.403284 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:01:35.403245    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d093745_9f02_40b2_a5fe_3eef57e1fb6a.slice/crio-255a6421783c81481d4cd738470fd43254c3e680912ae9436ef0f8096cba3268 WatchSource:0}: Error finding container 255a6421783c81481d4cd738470fd43254c3e680912ae9436ef0f8096cba3268: Status 404 returned error can't find the container with id 255a6421783c81481d4cd738470fd43254c3e680912ae9436ef0f8096cba3268
Nov 02 15:01:35.405242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:35.405219    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:01:35.953459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:35.953419    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-6m9kg" event={"ID":"3d093745-9f02-40b2-a5fe-3eef57e1fb6a","Type":"ContainerStarted","Data":"255a6421783c81481d4cd738470fd43254c3e680912ae9436ef0f8096cba3268"}
Nov 02 15:01:39.972123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:39.972090    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-6m9kg" event={"ID":"3d093745-9f02-40b2-a5fe-3eef57e1fb6a","Type":"ContainerStarted","Data":"403ce0c4767e7fce40d25bd19f4e107e16d67078f09cd60a72c13bc418a606a8"}
Nov 02 15:01:40.088712 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:40.088599    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-6m9kg" podStartSLOduration=1.6915389429999999 podStartE2EDuration="6.088584765s" totalImagesPullingTime="4.397045822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:01:40.08750457 +0000 UTC m=+306.525406712" watchObservedRunningTime="2026-11-02 15:01:40.088584765 +0000 UTC m=+306.526486905"
Nov 02 15:01:45.422399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.422360    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:01:45.424557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.424536    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.476615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.476588    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 02 15:01:45.476713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.476685    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 02 15:01:45.526239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.526206    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9nhl2\""
Nov 02 15:01:45.526371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.526183    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 02 15:01:45.527604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.527586    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:01:45.541317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.541294    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb74x\" (UniqueName: \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.541412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.541342    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.642379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.642349    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb74x\" (UniqueName: \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.642533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.642418    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.643077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.643052    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.667320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.667291    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:01:45.703121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.703098    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb74x\" (UniqueName: \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\") pod \"limitador-limitador-74dfcfb894-kt6hz\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:45.735478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:45.735453    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:46.109965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:46.109874    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:01:46.113558 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:01:46.113522    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5769df84_5293_4bdd_ad47_97fca0e74a0e.slice/crio-6d3abe0b8f44d852caf27557d402327a7bbc7c1835b54f33f6aedeb017f1b77b WatchSource:0}: Error finding container 6d3abe0b8f44d852caf27557d402327a7bbc7c1835b54f33f6aedeb017f1b77b: Status 404 returned error can't find the container with id 6d3abe0b8f44d852caf27557d402327a7bbc7c1835b54f33f6aedeb017f1b77b
Nov 02 15:01:46.955084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:46.955048    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:46.958291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:46.958272    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:47.001011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.000977    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" event={"ID":"5769df84-5293-4bdd-ad47-97fca0e74a0e","Type":"ContainerStarted","Data":"6d3abe0b8f44d852caf27557d402327a7bbc7c1835b54f33f6aedeb017f1b77b"}
Nov 02 15:01:47.010775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.010744    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rnjnp\""
Nov 02 15:01:47.032774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.032738    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:47.054345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.054318    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdcjc\" (UniqueName: \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\") pod \"authorino-57f7f8659f-llvzp\" (UID: \"af367a2d-42ed-4d30-b8ce-395c73beab3f\") " pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:47.155138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.155096    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdcjc\" (UniqueName: \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\") pod \"authorino-57f7f8659f-llvzp\" (UID: \"af367a2d-42ed-4d30-b8ce-395c73beab3f\") " pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:47.220592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.220514    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdcjc\" (UniqueName: \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\") pod \"authorino-57f7f8659f-llvzp\" (UID: \"af367a2d-42ed-4d30-b8ce-395c73beab3f\") " pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:47.270297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.270247    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:47.638826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:47.638735    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:47.642334 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:01:47.642291    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf367a2d_42ed_4d30_b8ce_395c73beab3f.slice/crio-82f186313303675e0579419fc751d48e55d9ea97bd9a37bc9c8095bb3f7f8bda WatchSource:0}: Error finding container 82f186313303675e0579419fc751d48e55d9ea97bd9a37bc9c8095bb3f7f8bda: Status 404 returned error can't find the container with id 82f186313303675e0579419fc751d48e55d9ea97bd9a37bc9c8095bb3f7f8bda
Nov 02 15:01:48.007183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:48.007133    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-llvzp" event={"ID":"af367a2d-42ed-4d30-b8ce-395c73beab3f","Type":"ContainerStarted","Data":"82f186313303675e0579419fc751d48e55d9ea97bd9a37bc9c8095bb3f7f8bda"}
Nov 02 15:01:49.830581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:49.830496    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:01:49.833977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:49.833947    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:01:49.889205 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:49.889164    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:01:49.984207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:49.984155    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcllh\" (UniqueName: \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\") pod \"authorino-56c6c9ddd5-hvr64\" (UID: \"ad986080-2b03-436a-91b5-e647d88bd1c5\") " pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:01:50.019277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.019225    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" event={"ID":"5769df84-5293-4bdd-ad47-97fca0e74a0e","Type":"ContainerStarted","Data":"11a1305b4b638edce1ccf49434f5d05ee124f0ae32044a6117f934007eb4a08b"}
Nov 02 15:01:50.019467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.019388    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:01:50.085779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.085683    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcllh\" (UniqueName: \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\") pod \"authorino-56c6c9ddd5-hvr64\" (UID: \"ad986080-2b03-436a-91b5-e647d88bd1c5\") " pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:01:50.136824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.136765    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" podStartSLOduration=1.7777953850000001 podStartE2EDuration="5.136749225s" totalImagesPullingTime="3.35895384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:01:50.135860411 +0000 UTC m=+316.573762554" watchObservedRunningTime="2026-11-02 15:01:50.136749225 +0000 UTC m=+316.574651365"
Nov 02 15:01:50.150205 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.150161    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcllh\" (UniqueName: \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\") pod \"authorino-56c6c9ddd5-hvr64\" (UID: \"ad986080-2b03-436a-91b5-e647d88bd1c5\") " pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:01:50.447335 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.447290    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:01:50.825854 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:50.825824    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:01:50.829882 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:01:50.829845    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad986080_2b03_436a_91b5_e647d88bd1c5.slice/crio-bb1d8b2139c822dfb9e09dc37e2cbcdb423e7862f268a75eb9fde3ebdb819869 WatchSource:0}: Error finding container bb1d8b2139c822dfb9e09dc37e2cbcdb423e7862f268a75eb9fde3ebdb819869: Status 404 returned error can't find the container with id bb1d8b2139c822dfb9e09dc37e2cbcdb423e7862f268a75eb9fde3ebdb819869
Nov 02 15:01:51.024547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:51.024456    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" event={"ID":"ad986080-2b03-436a-91b5-e647d88bd1c5","Type":"ContainerStarted","Data":"bb1d8b2139c822dfb9e09dc37e2cbcdb423e7862f268a75eb9fde3ebdb819869"}
Nov 02 15:01:51.025836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:51.025811    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-llvzp" event={"ID":"af367a2d-42ed-4d30-b8ce-395c73beab3f","Type":"ContainerStarted","Data":"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"}
Nov 02 15:01:51.139993 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:51.139945    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-llvzp" podStartSLOduration=2.049381874 podStartE2EDuration="5.139929462s" totalImagesPullingTime="3.090547588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:01:51.139417993 +0000 UTC m=+317.577320135" watchObservedRunningTime="2026-11-02 15:01:51.139929462 +0000 UTC m=+317.577831602"
Nov 02 15:01:53.036172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.036131    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" event={"ID":"ad986080-2b03-436a-91b5-e647d88bd1c5","Type":"ContainerStarted","Data":"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"}
Nov 02 15:01:53.152760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.152696    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" podStartSLOduration=2.884353573 podStartE2EDuration="4.152678961s" totalImagesPullingTime="1.268325388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:01:53.15246821 +0000 UTC m=+319.590370339" watchObservedRunningTime="2026-11-02 15:01:53.152678961 +0000 UTC m=+319.590581103"
Nov 02 15:01:53.191107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.191071    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:53.191323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.191268    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-llvzp" podUID="af367a2d-42ed-4d30-b8ce-395c73beab3f" containerName="authorino" containerID="cri-o://aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2" gracePeriod=30
Nov 02 15:01:53.448155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.448125    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:53.519231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.519171    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\" (UniqueName: \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\") pod \"af367a2d-42ed-4d30-b8ce-395c73beab3f\" (UID: \"af367a2d-42ed-4d30-b8ce-395c73beab3f\") "
Nov 02 15:01:53.521473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.521445    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc" pod "af367a2d-42ed-4d30-b8ce-395c73beab3f" (UID: "af367a2d-42ed-4d30-b8ce-395c73beab3f"). InnerVolumeSpecName "kube-api-access-fdcjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:01:53.620618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:53.620520    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdcjc\" (UniqueName: \"kubernetes.io/projected/af367a2d-42ed-4d30-b8ce-395c73beab3f-kube-api-access-fdcjc\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:01:54.041527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.041494    4470 generic.go:356] "Generic (PLEG): container finished" podID="af367a2d-42ed-4d30-b8ce-395c73beab3f" containerID="aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2" exitCode=0
Nov 02 15:01:54.041951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.041549    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-llvzp"
Nov 02 15:01:54.041951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.041584    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-llvzp" event={"ID":"af367a2d-42ed-4d30-b8ce-395c73beab3f","Type":"ContainerDied","Data":"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"}
Nov 02 15:01:54.041951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.041636    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-llvzp" event={"ID":"af367a2d-42ed-4d30-b8ce-395c73beab3f","Type":"ContainerDied","Data":"82f186313303675e0579419fc751d48e55d9ea97bd9a37bc9c8095bb3f7f8bda"}
Nov 02 15:01:54.041951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.041658    4470 scope.go:122] "RemoveContainer" containerID="aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"
Nov 02 15:01:54.050087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.050059    4470 scope.go:122] "RemoveContainer" containerID="aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"
Nov 02 15:01:54.050392 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:01:54.050372    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2\": container with ID starting with aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2 not found: ID does not exist" containerID="aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"
Nov 02 15:01:54.050460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.050407    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2"} err="failed to get container status \"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2\": rpc error: code = NotFound desc = could not find container \"aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2\": container with ID starting with aeac6de20aa43e1fc6243a0cd33673a2cde15c1466401024b076216cc6e0d7c2 not found: ID does not exist"
Nov 02 15:01:54.207484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.207446    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:54.256610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:54.256579    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-llvzp"]
Nov 02 15:01:55.703807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:55.703774    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af367a2d-42ed-4d30-b8ce-395c73beab3f" path="/var/lib/kubelet/pods/af367a2d-42ed-4d30-b8ce-395c73beab3f/volumes"
Nov 02 15:01:59.497580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.497540    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"]
Nov 02 15:01:59.498062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.497900    4470 cpu_manager.go:414] "containerMap: removing container" podUID="af367a2d-42ed-4d30-b8ce-395c73beab3f" containerName="authorino"
Nov 02 15:01:59.498062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.497911    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af367a2d-42ed-4d30-b8ce-395c73beab3f" containerName="authorino"
Nov 02 15:01:59.498062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.497976    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="af367a2d-42ed-4d30-b8ce-395c73beab3f" containerName="authorino"
Nov 02 15:01:59.502427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.502406    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.555306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.555271    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 02 15:01:59.604117 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.604081    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-b8tfw\""
Nov 02 15:01:59.604411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.604393    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 02 15:01:59.604722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.604705    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 02 15:01:59.604888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.604870    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 02 15:01:59.604970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.604874    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 02 15:01:59.606259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.606240    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"]
Nov 02 15:01:59.673279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.673240    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfm9q\" (UniqueName: \"kubernetes.io/projected/d47dea40-e602-4275-8be4-7d2d28eed346-kube-api-access-qfm9q\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.673463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.673303    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-metrics-certs\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.673463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.673397    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d47dea40-e602-4275-8be4-7d2d28eed346-manager-config\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.673463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.673426    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-cert\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.774885 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.774792    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfm9q\" (UniqueName: \"kubernetes.io/projected/d47dea40-e602-4275-8be4-7d2d28eed346-kube-api-access-qfm9q\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.774885 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.774847    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-metrics-certs\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.774885 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.774888    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d47dea40-e602-4275-8be4-7d2d28eed346-manager-config\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.775224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.774909    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-cert\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.775661 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.775633    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d47dea40-e602-4275-8be4-7d2d28eed346-manager-config\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.777564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.777542    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-cert\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.777659 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.777569    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d47dea40-e602-4275-8be4-7d2d28eed346-metrics-certs\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:01:59.836740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:01:59.836701    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfm9q\" (UniqueName: \"kubernetes.io/projected/d47dea40-e602-4275-8be4-7d2d28eed346-kube-api-access-qfm9q\") pod \"jobset-controller-manager-d85679894-74psm\" (UID: \"d47dea40-e602-4275-8be4-7d2d28eed346\") " pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:02:00.114037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:00.113939    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:02:00.490206 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:00.490153    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"]
Nov 02 15:02:00.493471 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:02:00.493443    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47dea40_e602_4275_8be4_7d2d28eed346.slice/crio-c48cdc41e879b6c57d6e5604f22c8e1359afdacd010466b51cee2f665f520c25 WatchSource:0}: Error finding container c48cdc41e879b6c57d6e5604f22c8e1359afdacd010466b51cee2f665f520c25: Status 404 returned error can't find the container with id c48cdc41e879b6c57d6e5604f22c8e1359afdacd010466b51cee2f665f520c25
Nov 02 15:02:01.027179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:01.027148    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:02:01.076856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:01.076819    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm" event={"ID":"d47dea40-e602-4275-8be4-7d2d28eed346","Type":"ContainerStarted","Data":"c48cdc41e879b6c57d6e5604f22c8e1359afdacd010466b51cee2f665f520c25"}
Nov 02 15:02:03.573078 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:03.573036    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:02:04.683454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:04.683415    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:02:04.683923 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:04.683651    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" podUID="5769df84-5293-4bdd-ad47-97fca0e74a0e" containerName="limitador" containerID="cri-o://11a1305b4b638edce1ccf49434f5d05ee124f0ae32044a6117f934007eb4a08b" gracePeriod=30
Nov 02 15:02:05.096997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.096961    4470 generic.go:356] "Generic (PLEG): container finished" podID="5769df84-5293-4bdd-ad47-97fca0e74a0e" containerID="11a1305b4b638edce1ccf49434f5d05ee124f0ae32044a6117f934007eb4a08b" exitCode=0
Nov 02 15:02:05.097183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.097037    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" event={"ID":"5769df84-5293-4bdd-ad47-97fca0e74a0e","Type":"ContainerDied","Data":"11a1305b4b638edce1ccf49434f5d05ee124f0ae32044a6117f934007eb4a08b"}
Nov 02 15:02:05.098540 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.098519    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm" event={"ID":"d47dea40-e602-4275-8be4-7d2d28eed346","Type":"ContainerStarted","Data":"5a64fbadc44ec330e9deb4a2b1b67ae505d792336dce5f248ae95ee8a6f4ef92"}
Nov 02 15:02:05.098664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.098630    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:02:05.216042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.215975    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm" podStartSLOduration=2.229596576 podStartE2EDuration="6.215953168s" totalImagesPullingTime="3.986356592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:02:05.215165872 +0000 UTC m=+331.653068219" watchObservedRunningTime="2026-11-02 15:02:05.215953168 +0000 UTC m=+331.653855309"
Nov 02 15:02:05.622798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.622773    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:02:05.729423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.729390    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\" (UniqueName: \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\") pod \"5769df84-5293-4bdd-ad47-97fca0e74a0e\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") "
Nov 02 15:02:05.729817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.729458    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\" (UniqueName: \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\") pod \"5769df84-5293-4bdd-ad47-97fca0e74a0e\" (UID: \"5769df84-5293-4bdd-ad47-97fca0e74a0e\") "
Nov 02 15:02:05.729860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.729828    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file" pod "5769df84-5293-4bdd-ad47-97fca0e74a0e" (UID: "5769df84-5293-4bdd-ad47-97fca0e74a0e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:02:05.731562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.731534    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x" pod "5769df84-5293-4bdd-ad47-97fca0e74a0e" (UID: "5769df84-5293-4bdd-ad47-97fca0e74a0e"). InnerVolumeSpecName "kube-api-access-mb74x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:02:05.830812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.830703    4470 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5769df84-5293-4bdd-ad47-97fca0e74a0e-config-file\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:02:05.830812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:05.830734    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb74x\" (UniqueName: \"kubernetes.io/projected/5769df84-5293-4bdd-ad47-97fca0e74a0e-kube-api-access-mb74x\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:02:06.103704 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:06.103627    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"
Nov 02 15:02:06.103900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:06.103628    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-kt6hz" event={"ID":"5769df84-5293-4bdd-ad47-97fca0e74a0e","Type":"ContainerDied","Data":"6d3abe0b8f44d852caf27557d402327a7bbc7c1835b54f33f6aedeb017f1b77b"}
Nov 02 15:02:06.103900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:06.103778    4470 scope.go:122] "RemoveContainer" containerID="11a1305b4b638edce1ccf49434f5d05ee124f0ae32044a6117f934007eb4a08b"
Nov 02 15:02:06.268108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:06.268069    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:02:06.317469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:06.317434    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-kt6hz"]
Nov 02 15:02:07.704202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:07.704152    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5769df84-5293-4bdd-ad47-97fca0e74a0e" path="/var/lib/kubelet/pods/5769df84-5293-4bdd-ad47-97fca0e74a0e/volumes"
Nov 02 15:02:09.028984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.028944    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"]
Nov 02 15:02:09.029386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.029345    4470 cpu_manager.go:414] "containerMap: removing container" podUID="5769df84-5293-4bdd-ad47-97fca0e74a0e" containerName="limitador"
Nov 02 15:02:09.029386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.029357    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5769df84-5293-4bdd-ad47-97fca0e74a0e" containerName="limitador"
Nov 02 15:02:09.029457 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.029420    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="5769df84-5293-4bdd-ad47-97fca0e74a0e" containerName="limitador"
Nov 02 15:02:09.034557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.034538    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.091585 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.091552    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"]
Nov 02 15:02:09.139337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.139301    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 15:02:09.139506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.139391    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 15:02:09.158406 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.158368    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6256836a-d15d-4a91-a040-3807456115b3-data\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.158572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.158416    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m94tx\" (UniqueName: \"kubernetes.io/projected/6256836a-d15d-4a91-a040-3807456115b3-kube-api-access-m94tx\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.184119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.184079    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 02 15:02:09.184119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.184093    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-92c86\""
Nov 02 15:02:09.259285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.259250    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6256836a-d15d-4a91-a040-3807456115b3-data\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.259285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.259289    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m94tx\" (UniqueName: \"kubernetes.io/projected/6256836a-d15d-4a91-a040-3807456115b3-kube-api-access-m94tx\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.259664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.259643    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6256836a-d15d-4a91-a040-3807456115b3-data\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.320556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.320476    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m94tx\" (UniqueName: \"kubernetes.io/projected/6256836a-d15d-4a91-a040-3807456115b3-kube-api-access-m94tx\") pod \"postgres-7cdbf6dc44-2srjn\" (UID: \"6256836a-d15d-4a91-a040-3807456115b3\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.351513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.351478    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:09.721602 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:09.721574    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"]
Nov 02 15:02:09.724989 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:02:09.724948    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6256836a_d15d_4a91_a040_3807456115b3.slice/crio-ce6596cbcc8a4c718dcbf15c3a5934372948d55bb42618daac43d2eb80db5903 WatchSource:0}: Error finding container ce6596cbcc8a4c718dcbf15c3a5934372948d55bb42618daac43d2eb80db5903: Status 404 returned error can't find the container with id ce6596cbcc8a4c718dcbf15c3a5934372948d55bb42618daac43d2eb80db5903
Nov 02 15:02:10.121958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:10.121868    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn" event={"ID":"6256836a-d15d-4a91-a040-3807456115b3","Type":"ContainerStarted","Data":"ce6596cbcc8a4c718dcbf15c3a5934372948d55bb42618daac43d2eb80db5903"}
Nov 02 15:02:16.109085 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:16.109043    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-d85679894-74psm"
Nov 02 15:02:16.152748 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:16.152711    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn" event={"ID":"6256836a-d15d-4a91-a040-3807456115b3","Type":"ContainerStarted","Data":"54d32ce69e7ada59b019bbbc49c71d9b824d2b98937316d6cb140e4a71a6ed69"}
Nov 02 15:02:16.152933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:16.152861    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:16.366686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:16.366624    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn" podStartSLOduration=2.160603973 podStartE2EDuration="8.36660998s" totalImagesPullingTime="6.206006007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:02:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:02:16.365879262 +0000 UTC m=+342.803781404" watchObservedRunningTime="2026-11-02 15:02:16.36660998 +0000 UTC m=+342.804512120"
Nov 02 15:02:22.186180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:22.186152    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-2srjn"
Nov 02 15:02:30.575683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.575646    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:02:30.579224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.579179    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.631789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.631748    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 15:02:30.632277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.632216    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 15:02:30.632394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.632288    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-77ggk\""
Nov 02 15:02:30.634719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.634683    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmjf\" (UniqueName: \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.634719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.634725    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.634934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.634765    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.681451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.681416    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 15:02:30.681451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.681442    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 15:02:30.681691 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.681566    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 15:02:30.683076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.683057    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:02:30.735813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.735774    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmjf\" (UniqueName: \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.735813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.735813    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.736084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.735839    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.738441 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.738412    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.738563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.738466    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.796676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.796643    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmjf\" (UniqueName: \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\") pod \"maas-controller-76976749fb-92m7t\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") " pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:30.892577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:30.892490    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:31.316113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:31.316081    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:02:31.318665 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:02:31.318633    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e9527e_58d7_4ecc_9cf8_81a0cc60a792.slice/crio-3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560 WatchSource:0}: Error finding container 3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560: Status 404 returned error can't find the container with id 3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560
Nov 02 15:02:32.217761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:32.217724    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-92m7t" event={"ID":"23e9527e-58d7-4ecc-9cf8-81a0cc60a792","Type":"ContainerStarted","Data":"3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560"}
Nov 02 15:02:33.569150 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:33.569103    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:02:36.299826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:36.299790    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 15:02:37.241679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:37.241643    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-92m7t" event={"ID":"23e9527e-58d7-4ecc-9cf8-81a0cc60a792","Type":"ContainerStarted","Data":"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"}
Nov 02 15:02:37.241869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:37.241785    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:37.363273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:37.363205    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-76976749fb-92m7t" podStartSLOduration=2.435148887 podStartE2EDuration="7.363172193s" totalImagesPullingTime="4.928023306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:02:37.362529994 +0000 UTC m=+363.800432136" watchObservedRunningTime="2026-11-02 15:02:37.363172193 +0000 UTC m=+363.801074336"
Nov 02 15:02:39.628707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.628665    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"]
Nov 02 15:02:39.632760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.632739    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.684768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.684729    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 02 15:02:39.684768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.684731    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 02 15:02:39.719570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.719537    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.719753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.719607    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.719818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.719788    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkm8n\" (UniqueName: \"kubernetes.io/projected/87736579-2a3c-48d5-b5fa-6c39c2726194-kube-api-access-vkm8n\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.719857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.719838    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.734292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.734259    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-9s89s\""
Nov 02 15:02:39.734292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.734260    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 02 15:02:39.735982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.735956    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"]
Nov 02 15:02:39.820804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.820763    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkm8n\" (UniqueName: \"kubernetes.io/projected/87736579-2a3c-48d5-b5fa-6c39c2726194-kube-api-access-vkm8n\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.821013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.820821    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.821013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.820871    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.821013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.820926    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.821174 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:39.821027    4470 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 02 15:02:39.821174 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:39.821073    4470 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 02 15:02:39.821174 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:39.821093    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source podName:87736579-2a3c-48d5-b5fa-6c39c2726194 nodeName:}" failed. No retries permitted until 2026-11-02 15:02:40.32106988 +0000 UTC m=+366.758972020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-4v492" (UID: "87736579-2a3c-48d5-b5fa-6c39c2726194") : configmap "usage-tenancy-proxy-source" not found
Nov 02 15:02:39.821174 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:02:39.821155    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert podName:87736579-2a3c-48d5-b5fa-6c39c2726194 nodeName:}" failed. No retries permitted until 2026-11-02 15:02:40.321136097 +0000 UTC m=+366.759038222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-4v492" (UID: "87736579-2a3c-48d5-b5fa-6c39c2726194") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 02 15:02:39.821785 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.821753    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:39.884535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:39.884451    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkm8n\" (UniqueName: \"kubernetes.io/projected/87736579-2a3c-48d5-b5fa-6c39c2726194-kube-api-access-vkm8n\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.325755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.325711    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.325946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.325771    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.326581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.326560    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/87736579-2a3c-48d5-b5fa-6c39c2726194-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.328107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.328085    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/87736579-2a3c-48d5-b5fa-6c39c2726194-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-4v492\" (UID: \"87736579-2a3c-48d5-b5fa-6c39c2726194\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.546496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.546452    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:40.920116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:40.920092    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"]
Nov 02 15:02:40.922639 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:02:40.922609    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87736579_2a3c_48d5_b5fa_6c39c2726194.slice/crio-df162e2abb92e7a8699afcc5416e13bb1e9cec82414c77d6bedec2d1fd98fd64 WatchSource:0}: Error finding container df162e2abb92e7a8699afcc5416e13bb1e9cec82414c77d6bedec2d1fd98fd64: Status 404 returned error can't find the container with id df162e2abb92e7a8699afcc5416e13bb1e9cec82414c77d6bedec2d1fd98fd64
Nov 02 15:02:41.260826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:41.260791    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492" event={"ID":"87736579-2a3c-48d5-b5fa-6c39c2726194","Type":"ContainerStarted","Data":"df162e2abb92e7a8699afcc5416e13bb1e9cec82414c77d6bedec2d1fd98fd64"}
Nov 02 15:02:48.249917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:48.249881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:02:54.324562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:54.324525    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492" event={"ID":"87736579-2a3c-48d5-b5fa-6c39c2726194","Type":"ContainerStarted","Data":"961ee0c78d5763ec501fbcf048ba2c48634398cc9725a02c1f76246187ea995c"}
Nov 02 15:02:54.324957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:54.324645    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:02:54.441953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:02:54.441896    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492" podStartSLOduration=2.877330209 podStartE2EDuration="15.441879702s" totalImagesPullingTime="12.564549493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:02:54.440453132 +0000 UTC m=+380.878355274" watchObservedRunningTime="2026-11-02 15:02:54.441879702 +0000 UTC m=+380.879781842"
Nov 02 15:03:03.564979 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:03.564938    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:03:05.334469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:05.334430    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4v492"
Nov 02 15:03:10.239463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.239427    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:03:10.246907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.246882    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301346    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301402    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301489    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301514    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301534    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301613    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pvwd\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.301699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.301647    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.352556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.352520    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:03:10.402391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402356    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402416    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402457    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402484    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402516    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402747    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pvwd\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.402863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.402839    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.403249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.403214    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.405058 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.405035    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.405167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.405073    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.405167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.405085    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.405283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.405264    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.471344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.471312    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.471807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.471784    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pvwd\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-g5twx\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.559523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.559444    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:10.988665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.988630    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:03:10.991314 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:03:10.991281    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513f0dec_6b25_4ad9_a6bb_5bec7b18abb5.slice/crio-81a35feb28d7ae0f1d0a02ecf6cb5eff9a93b17ef7b71b6985dc56f78af0d635 WatchSource:0}: Error finding container 81a35feb28d7ae0f1d0a02ecf6cb5eff9a93b17ef7b71b6985dc56f78af0d635: Status 404 returned error can't find the container with id 81a35feb28d7ae0f1d0a02ecf6cb5eff9a93b17ef7b71b6985dc56f78af0d635
Nov 02 15:03:10.993601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.993565    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:03:10.993684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:10.993632    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:03:11.398090 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:11.398003    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" event={"ID":"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5","Type":"ContainerStarted","Data":"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"}
Nov 02 15:03:11.398090 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:11.398067    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" event={"ID":"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5","Type":"ContainerStarted","Data":"81a35feb28d7ae0f1d0a02ecf6cb5eff9a93b17ef7b71b6985dc56f78af0d635"}
Nov 02 15:03:11.398527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:11.398097    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:11.569255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:11.569198    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" podStartSLOduration=1.569170802 podStartE2EDuration="1.569170802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:03:11.518988635 +0000 UTC m=+397.956890775" watchObservedRunningTime="2026-11-02 15:03:11.569170802 +0000 UTC m=+398.007072942"
Nov 02 15:03:12.403562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:12.403524    4470 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-g5twx container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:03:12.403949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:12.403582    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:03:15.402965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.402933    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:03:15.631652 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.631613    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:03:15.631948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.631912    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerName="discovery" containerID="cri-o://5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1" gracePeriod=30
Nov 02 15:03:15.890495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.890468    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:03:15.959250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959145    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959221    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959245    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959277    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959317    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959369    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959441    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\") pod \"5fdc343b-6af9-48da-a0a4-654450468e6b\" (UID: \"5fdc343b-6af9-48da-a0a4-654450468e6b\") "
Nov 02 15:03:15.959750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959653    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:03:15.959803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.959745    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-ca-configmap\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:15.961727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.961694    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:03:15.961727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.961718    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:03:15.961968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.961942    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:03:15.962028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.961946    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "kube-api-access-grkfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:03:15.962028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.962003    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:03:15.962137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:15.962078    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig" pod "5fdc343b-6af9-48da-a0a4-654450468e6b" (UID: "5fdc343b-6af9-48da-a0a4-654450468e6b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:03:16.060476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060436    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grkfz\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-kube-api-access-grkfz\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.060476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060468    4470 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-cacerts\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.060476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060479    4470 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-kubeconfig\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.060696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060488    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.060696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060498    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5fdc343b-6af9-48da-a0a4-654450468e6b-istio-csr-dns-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.060696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.060506    4470 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fdc343b-6af9-48da-a0a4-654450468e6b-local-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:16.114973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.114937    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-62ftm"]
Nov 02 15:03:16.115329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.115314    4470 cpu_manager.go:414] "containerMap: removing container" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerName="discovery"
Nov 02 15:03:16.115398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.115331    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerName="discovery"
Nov 02 15:03:16.115434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.115404    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerName="discovery"
Nov 02 15:03:16.118696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.118678    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.161851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.161810    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-cert\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.162064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.161957    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-metrics-certs\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.162064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.162030    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bz54\" (UniqueName: \"kubernetes.io/projected/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-kube-api-access-9bz54\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.171291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.171267    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 02 15:03:16.171291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.171287    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 02 15:03:16.171478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.171292    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zzf6r\""
Nov 02 15:03:16.177931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.177891    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-62ftm"]
Nov 02 15:03:16.263509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.263429    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-metrics-certs\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.263509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.263469    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bz54\" (UniqueName: \"kubernetes.io/projected/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-kube-api-access-9bz54\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.263509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.263502    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-cert\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.266017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.265982    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-cert\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.266137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.266024    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-metrics-certs\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.323742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.323710    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bz54\" (UniqueName: \"kubernetes.io/projected/f5b0ee7b-b9b4-4029-9cad-532b3c334a5b-kube-api-access-9bz54\") pod \"odh-model-controller-769875db56-62ftm\" (UID: \"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b\") " pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.424870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.424656    4470 generic.go:356] "Generic (PLEG): container finished" podID="5fdc343b-6af9-48da-a0a4-654450468e6b" containerID="5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1" exitCode=0
Nov 02 15:03:16.424870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.424707    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" event={"ID":"5fdc343b-6af9-48da-a0a4-654450468e6b","Type":"ContainerDied","Data":"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"}
Nov 02 15:03:16.424870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.424741    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"
Nov 02 15:03:16.424870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.424758    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq" event={"ID":"5fdc343b-6af9-48da-a0a4-654450468e6b","Type":"ContainerDied","Data":"faed95c92de3a1b8f1084dbac58a06b917de2b2ee5f38c59bbf019cda5d18c23"}
Nov 02 15:03:16.424870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.424790    4470 scope.go:122] "RemoveContainer" containerID="5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"
Nov 02 15:03:16.432684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.432657    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:16.434118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.434093    4470 scope.go:122] "RemoveContainer" containerID="5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"
Nov 02 15:03:16.434475 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:16.434424    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1\": container with ID starting with 5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1 not found: ID does not exist" containerID="5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"
Nov 02 15:03:16.434475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.434461    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1"} err="failed to get container status \"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1\": rpc error: code = NotFound desc = could not find container \"5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1\": container with ID starting with 5f9a29eadc989309660bff6fea5544c88390dc03d3c0ec1fdc6169c3e2c19ba1 not found: ID does not exist"
Nov 02 15:03:16.589383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.589296    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:03:16.594012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.593984    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-klqrq"]
Nov 02 15:03:16.803965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:16.803927    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-62ftm"]
Nov 02 15:03:16.807106 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:03:16.807078    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b0ee7b_b9b4_4029_9cad_532b3c334a5b.slice/crio-c37c031d83bdd07420cedc66f8031f6e06bf9fec42900e3f8ca41d6547b7786c WatchSource:0}: Error finding container c37c031d83bdd07420cedc66f8031f6e06bf9fec42900e3f8ca41d6547b7786c: Status 404 returned error can't find the container with id c37c031d83bdd07420cedc66f8031f6e06bf9fec42900e3f8ca41d6547b7786c
Nov 02 15:03:17.431148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:17.431110    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-62ftm" event={"ID":"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b","Type":"ContainerStarted","Data":"c37c031d83bdd07420cedc66f8031f6e06bf9fec42900e3f8ca41d6547b7786c"}
Nov 02 15:03:17.706410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:17.706380    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fdc343b-6af9-48da-a0a4-654450468e6b" path="/var/lib/kubelet/pods/5fdc343b-6af9-48da-a0a4-654450468e6b/volumes"
Nov 02 15:03:20.446909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:20.446875    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-62ftm" event={"ID":"f5b0ee7b-b9b4-4029-9cad-532b3c334a5b","Type":"ContainerStarted","Data":"866ee9dd9fc4f66f5d3a7cc3736888ae3975fe886c2df2cb6ec30b415c860146"}
Nov 02 15:03:20.447371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:20.446993    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:20.564503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:20.564441    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-62ftm" podStartSLOduration=1.397564046 podStartE2EDuration="4.564422245s" totalImagesPullingTime="3.166858199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:03:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:03:20.563853945 +0000 UTC m=+407.001756087" watchObservedRunningTime="2026-11-02 15:03:20.564422245 +0000 UTC m=+407.002324386"
Nov 02 15:03:23.811370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:23.811329    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-vl42f"]
Nov 02 15:03:24.027313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.027270    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-vl42f"]
Nov 02 15:03:24.027313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.027311    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 15:03:24.027534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.027389    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.048168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.048140    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.091145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.091066    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 15:03:24.135705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.135673    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zvk6\" (UniqueName: \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.135879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.135743    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc86r\" (UniqueName: \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\") pod \"authorino-6f6944bcdd-vl42f\" (UID: \"e5012b97-848c-4d29-a52f-1689dc6456b5\") " pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.135879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.135833    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.135879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.135865    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.135987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.135899    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.138883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.138847    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-vkjdq\""
Nov 02 15:03:24.138883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.138855    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 15:03:24.139104 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.138933    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 15:03:24.211687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.211649    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-vl42f"]
Nov 02 15:03:24.211892 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:24.211854    4470 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xc86r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-vl42f" podUID="e5012b97-848c-4d29-a52f-1689dc6456b5"
Nov 02 15:03:24.236637 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.236597    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc86r\" (UniqueName: \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\") pod \"authorino-6f6944bcdd-vl42f\" (UID: \"e5012b97-848c-4d29-a52f-1689dc6456b5\") " pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.236807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.236671    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.236807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.236699    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.236807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.236718    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.236807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.236769    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zvk6\" (UniqueName: \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.237455 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.237421    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.239158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.239136    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.239277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.239180    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.243328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.243304    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:24.264227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.264173    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:24.298212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.298149    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc86r\" (UniqueName: \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\") pod \"authorino-6f6944bcdd-vl42f\" (UID: \"e5012b97-848c-4d29-a52f-1689dc6456b5\") " pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.298354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.298320    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zvk6\" (UniqueName: \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\") pod \"maas-api-88df9b747-twfbl\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") " pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.305286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.305257    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:24.337562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.337525    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24vb4\" (UniqueName: \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\") pod \"authorino-5dcfdbc796-fg2c2\" (UID: \"c36e7e27-a197-4460-b1ed-9c920d43ff8d\") " pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:24.379459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.379371    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:24.438411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.438377    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24vb4\" (UniqueName: \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\") pod \"authorino-5dcfdbc796-fg2c2\" (UID: \"c36e7e27-a197-4460-b1ed-9c920d43ff8d\") " pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:24.472223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.465720    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.497083 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.497050    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:24.499174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.499153    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24vb4\" (UniqueName: \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\") pod \"authorino-5dcfdbc796-fg2c2\" (UID: \"c36e7e27-a197-4460-b1ed-9c920d43ff8d\") " pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:24.539139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.539105    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\" (UniqueName: \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\") pod \"e5012b97-848c-4d29-a52f-1689dc6456b5\" (UID: \"e5012b97-848c-4d29-a52f-1689dc6456b5\") "
Nov 02 15:03:24.541293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.541266    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r" pod "e5012b97-848c-4d29-a52f-1689dc6456b5" (UID: "e5012b97-848c-4d29-a52f-1689dc6456b5"). InnerVolumeSpecName "kube-api-access-xc86r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:03:24.593093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.593057    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:24.623656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.623620    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:24.639999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.639923    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xc86r\" (UniqueName: \"kubernetes.io/projected/e5012b97-848c-4d29-a52f-1689dc6456b5-kube-api-access-xc86r\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:24.778407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.778364    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 15:03:24.782760 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:03:24.782732    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b5032f_6be1_400d_b463_3007837e608b.slice/crio-590dba5e87347a19773e25d9f63c4894165b55c8bb888a5b6550a6dddfd0d6e4 WatchSource:0}: Error finding container 590dba5e87347a19773e25d9f63c4894165b55c8bb888a5b6550a6dddfd0d6e4: Status 404 returned error can't find the container with id 590dba5e87347a19773e25d9f63c4894165b55c8bb888a5b6550a6dddfd0d6e4
Nov 02 15:03:24.972798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:24.972763    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:24.976298 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:03:24.976267    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc36e7e27_a197_4460_b1ed_9c920d43ff8d.slice/crio-07756624bfb0cb34c733a1aad661eed11d97e0aa45959d39f567676131c4eaf1 WatchSource:0}: Error finding container 07756624bfb0cb34c733a1aad661eed11d97e0aa45959d39f567676131c4eaf1: Status 404 returned error can't find the container with id 07756624bfb0cb34c733a1aad661eed11d97e0aa45959d39f567676131c4eaf1
Nov 02 15:03:25.471503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:25.471459    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" event={"ID":"c36e7e27-a197-4460-b1ed-9c920d43ff8d","Type":"ContainerStarted","Data":"07756624bfb0cb34c733a1aad661eed11d97e0aa45959d39f567676131c4eaf1"}
Nov 02 15:03:25.472858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:25.472835    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-vl42f"
Nov 02 15:03:25.472997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:25.472830    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" event={"ID":"91b5032f-6be1-400d-b463-3007837e608b","Type":"ContainerStarted","Data":"590dba5e87347a19773e25d9f63c4894165b55c8bb888a5b6550a6dddfd0d6e4"}
Nov 02 15:03:25.696910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:25.696876    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-vl42f"]
Nov 02 15:03:25.704203 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:25.704162    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-vl42f"]
Nov 02 15:03:27.483789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.483757    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" podUID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" containerName="authorino" containerID="cri-o://7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52" gracePeriod=30
Nov 02 15:03:27.484264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.483760    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" event={"ID":"c36e7e27-a197-4460-b1ed-9c920d43ff8d","Type":"ContainerStarted","Data":"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"}
Nov 02 15:03:27.485544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.485513    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" event={"ID":"91b5032f-6be1-400d-b463-3007837e608b","Type":"ContainerStarted","Data":"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"}
Nov 02 15:03:27.485658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.485649    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:27.599873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.599767    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" podStartSLOduration=1.195537742 podStartE2EDuration="3.599750855s" totalImagesPullingTime="2.404213113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:03:27.59835994 +0000 UTC m=+414.036262082" watchObservedRunningTime="2026-11-02 15:03:27.599750855 +0000 UTC m=+414.037652995"
Nov 02 15:03:27.704653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.704622    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5012b97-848c-4d29-a52f-1689dc6456b5" path="/var/lib/kubelet/pods/e5012b97-848c-4d29-a52f-1689dc6456b5/volumes"
Nov 02 15:03:27.718463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.718408    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" podStartSLOduration=2.182453839 podStartE2EDuration="4.718392028s" totalImagesPullingTime="2.535938189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:03:27.718319919 +0000 UTC m=+414.156222060" watchObservedRunningTime="2026-11-02 15:03:27.718392028 +0000 UTC m=+414.156294169"
Nov 02 15:03:27.835935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.835907    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:27.974489 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.974455    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\" (UniqueName: \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\") pod \"c36e7e27-a197-4460-b1ed-9c920d43ff8d\" (UID: \"c36e7e27-a197-4460-b1ed-9c920d43ff8d\") "
Nov 02 15:03:27.976535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:27.976509    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4" pod "c36e7e27-a197-4460-b1ed-9c920d43ff8d" (UID: "c36e7e27-a197-4460-b1ed-9c920d43ff8d"). InnerVolumeSpecName "kube-api-access-24vb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:03:28.075953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.075919    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24vb4\" (UniqueName: \"kubernetes.io/projected/c36e7e27-a197-4460-b1ed-9c920d43ff8d-kube-api-access-24vb4\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:28.491242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.491202    4470 generic.go:356] "Generic (PLEG): container finished" podID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" containerID="7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52" exitCode=0
Nov 02 15:03:28.491682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.491259    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2"
Nov 02 15:03:28.491682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.491278    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" event={"ID":"c36e7e27-a197-4460-b1ed-9c920d43ff8d","Type":"ContainerDied","Data":"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"}
Nov 02 15:03:28.491682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.491317    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5dcfdbc796-fg2c2" event={"ID":"c36e7e27-a197-4460-b1ed-9c920d43ff8d","Type":"ContainerDied","Data":"07756624bfb0cb34c733a1aad661eed11d97e0aa45959d39f567676131c4eaf1"}
Nov 02 15:03:28.491682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.491335    4470 scope.go:122] "RemoveContainer" containerID="7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"
Nov 02 15:03:28.500592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.500571    4470 scope.go:122] "RemoveContainer" containerID="7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"
Nov 02 15:03:28.500868 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:28.500849    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52\": container with ID starting with 7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52 not found: ID does not exist" containerID="7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"
Nov 02 15:03:28.500912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.500880    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52"} err="failed to get container status \"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52\": rpc error: code = NotFound desc = could not find container \"7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52\": container with ID starting with 7aee51f4d2ed91a215de4db12f40d1bcec2128e8a7caa48f69599b1eb749ba52 not found: ID does not exist"
Nov 02 15:03:28.658499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.658378    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:28.662901 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:28.662859    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dcfdbc796-fg2c2"]
Nov 02 15:03:29.703969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:29.703936    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" path="/var/lib/kubelet/pods/c36e7e27-a197-4460-b1ed-9c920d43ff8d/volumes"
Nov 02 15:03:29.982307 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:29.982209    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:03:29.982528 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:29.982401    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" podUID="ad986080-2b03-436a-91b5-e647d88bd1c5" containerName="authorino" containerID="cri-o://56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9" gracePeriod=30
Nov 02 15:03:30.232369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.232302    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:03:30.297723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.297682    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\" (UniqueName: \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\") pod \"ad986080-2b03-436a-91b5-e647d88bd1c5\" (UID: \"ad986080-2b03-436a-91b5-e647d88bd1c5\") "
Nov 02 15:03:30.299803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.299772    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh" pod "ad986080-2b03-436a-91b5-e647d88bd1c5" (UID: "ad986080-2b03-436a-91b5-e647d88bd1c5"). InnerVolumeSpecName "kube-api-access-tcllh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:03:30.399526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.399484    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcllh\" (UniqueName: \"kubernetes.io/projected/ad986080-2b03-436a-91b5-e647d88bd1c5-kube-api-access-tcllh\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:03:30.502700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.502602    4470 generic.go:356] "Generic (PLEG): container finished" podID="ad986080-2b03-436a-91b5-e647d88bd1c5" containerID="56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9" exitCode=0
Nov 02 15:03:30.502700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.502650    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" event={"ID":"ad986080-2b03-436a-91b5-e647d88bd1c5","Type":"ContainerDied","Data":"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"}
Nov 02 15:03:30.502700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.502655    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64"
Nov 02 15:03:30.502700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.502686    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-hvr64" event={"ID":"ad986080-2b03-436a-91b5-e647d88bd1c5","Type":"ContainerDied","Data":"bb1d8b2139c822dfb9e09dc37e2cbcdb423e7862f268a75eb9fde3ebdb819869"}
Nov 02 15:03:30.502700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.502705    4470 scope.go:122] "RemoveContainer" containerID="56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"
Nov 02 15:03:30.512257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.512227    4470 scope.go:122] "RemoveContainer" containerID="56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"
Nov 02 15:03:30.512618 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:30.512595    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9\": container with ID starting with 56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9 not found: ID does not exist" containerID="56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"
Nov 02 15:03:30.512686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.512631    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9"} err="failed to get container status \"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9\": rpc error: code = NotFound desc = could not find container \"56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9\": container with ID starting with 56f33849f8736eb587601688a39156ff7e758296ed248b8e76dab469167598b9 not found: ID does not exist"
Nov 02 15:03:30.669083 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.669044    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:03:30.674521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:30.674494    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-hvr64"]
Nov 02 15:03:31.453665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:31.453630    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-62ftm"
Nov 02 15:03:31.704113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:31.704032    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad986080-2b03-436a-91b5-e647d88bd1c5" path="/var/lib/kubelet/pods/ad986080-2b03-436a-91b5-e647d88bd1c5/volumes"
Nov 02 15:03:33.497525 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:03:33.497497    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 15:03:33.564320 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:03:33.564286    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:04:03.564328 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:04:03.564288    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:04:33.564943 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:04:33.564889    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:05:03.564762 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:05:03.564719    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:05:26.383133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383088    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"]
Nov 02 15:05:26.383687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383634    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ad986080-2b03-436a-91b5-e647d88bd1c5" containerName="authorino"
Nov 02 15:05:26.383687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383654    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad986080-2b03-436a-91b5-e647d88bd1c5" containerName="authorino"
Nov 02 15:05:26.383687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383668    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" containerName="authorino"
Nov 02 15:05:26.383687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383675    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" containerName="authorino"
Nov 02 15:05:26.383891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383797    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad986080-2b03-436a-91b5-e647d88bd1c5" containerName="authorino"
Nov 02 15:05:26.383891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.383810    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c36e7e27-a197-4460-b1ed-9c920d43ff8d" containerName="authorino"
Nov 02 15:05:26.386524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.386499    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.488278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.488245    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 02 15:05:26.488278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.488248    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 15:05:26.488485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.488256    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 15:05:26.488485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.488453    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vgzsg\""
Nov 02 15:05:26.490047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.490027    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"]
Nov 02 15:05:26.518269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518237    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.518269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518271    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.518607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518314    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.518607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518417    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.518607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518444    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00969f6-55d8-46d4-b90e-6391709f1798-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.518607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.518540    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjqhp\" (UniqueName: \"kubernetes.io/projected/c00969f6-55d8-46d4-b90e-6391709f1798-kube-api-access-kjqhp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619288    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjqhp\" (UniqueName: \"kubernetes.io/projected/c00969f6-55d8-46d4-b90e-6391709f1798-kube-api-access-kjqhp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619343    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619368    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619420    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619464    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619498    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00969f6-55d8-46d4-b90e-6391709f1798-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619880    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619884    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.619992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.619924    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.621633 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.621614    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c00969f6-55d8-46d4-b90e-6391709f1798-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.621898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.621878    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c00969f6-55d8-46d4-b90e-6391709f1798-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.682150 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.682068    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjqhp\" (UniqueName: \"kubernetes.io/projected/c00969f6-55d8-46d4-b90e-6391709f1798-kube-api-access-kjqhp\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2\" (UID: \"c00969f6-55d8-46d4-b90e-6391709f1798\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:26.698951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:26.698910    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:27.080070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:27.079944    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"]
Nov 02 15:05:27.083033 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:27.083007    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00969f6_55d8_46d4_b90e_6391709f1798.slice/crio-8031b2d4dcc0c4074471299bb7c4e6163647d0a7e76ba362ceaf6b2caec97211 WatchSource:0}: Error finding container 8031b2d4dcc0c4074471299bb7c4e6163647d0a7e76ba362ceaf6b2caec97211: Status 404 returned error can't find the container with id 8031b2d4dcc0c4074471299bb7c4e6163647d0a7e76ba362ceaf6b2caec97211
Nov 02 15:05:27.998984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:27.998945    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2" event={"ID":"c00969f6-55d8-46d4-b90e-6391709f1798","Type":"ContainerStarted","Data":"8031b2d4dcc0c4074471299bb7c4e6163647d0a7e76ba362ceaf6b2caec97211"}
Nov 02 15:05:29.005070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:29.005030    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2" event={"ID":"c00969f6-55d8-46d4-b90e-6391709f1798","Type":"ContainerStarted","Data":"668e239017b00e396ba60d545a6d478c58f5985a206192dc855af94f8d3fe2d1"}
Nov 02 15:05:29.125555 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:29.125497    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2" podStartSLOduration=1.6220070359999998 podStartE2EDuration="3.12548181s" totalImagesPullingTime="1.503474774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:05:29.124303642 +0000 UTC m=+535.562205783" watchObservedRunningTime="2026-11-02 15:05:29.12548181 +0000 UTC m=+535.563383951"
Nov 02 15:05:29.703713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:29.703678    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:30.797713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.797679    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-fxkf2"]
Nov 02 15:05:30.801273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.801251    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.853558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.853525    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Nov 02 15:05:30.904285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.904254    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-fxkf2"]
Nov 02 15:05:30.959384 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959352    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.959563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959392    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.959563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959463    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwfz8\" (UniqueName: \"kubernetes.io/projected/b51259a2-7ada-4385-82b0-3586cfdb59ba-kube-api-access-nwfz8\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.959563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959528    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51259a2-7ada-4385-82b0-3586cfdb59ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.959738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959624    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:30.959738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:30.959658    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.060895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060801    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.060895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060841    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.060895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060883    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060917    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060941    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwfz8\" (UniqueName: \"kubernetes.io/projected/b51259a2-7ada-4385-82b0-3586cfdb59ba-kube-api-access-nwfz8\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.060992    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51259a2-7ada-4385-82b0-3586cfdb59ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.061303    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.061369    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.061532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.061493    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-home\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.063139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.063118    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b51259a2-7ada-4385-82b0-3586cfdb59ba-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.063541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.063519    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b51259a2-7ada-4385-82b0-3586cfdb59ba-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.121766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.121734    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwfz8\" (UniqueName: \"kubernetes.io/projected/b51259a2-7ada-4385-82b0-3586cfdb59ba-kube-api-access-nwfz8\") pod \"echo-simulator-kserve-64b6f8f96c-fxkf2\" (UID: \"b51259a2-7ada-4385-82b0-3586cfdb59ba\") " pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.413825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.413730    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:31.790407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:31.790372    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-fxkf2"]
Nov 02 15:05:31.793513 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:31.793479    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb51259a2_7ada_4385_82b0_3586cfdb59ba.slice/crio-52f85f79e4d7f627f19721bbcbfa8d74e9948ce1f90b29355af2107fded58198 WatchSource:0}: Error finding container 52f85f79e4d7f627f19721bbcbfa8d74e9948ce1f90b29355af2107fded58198: Status 404 returned error can't find the container with id 52f85f79e4d7f627f19721bbcbfa8d74e9948ce1f90b29355af2107fded58198
Nov 02 15:05:32.019259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:32.019220    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2" event={"ID":"b51259a2-7ada-4385-82b0-3586cfdb59ba","Type":"ContainerStarted","Data":"52f85f79e4d7f627f19721bbcbfa8d74e9948ce1f90b29355af2107fded58198"}
Nov 02 15:05:33.564560 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:05:33.564510    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:05:35.038201 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:35.038151    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2" event={"ID":"b51259a2-7ada-4385-82b0-3586cfdb59ba","Type":"ContainerStarted","Data":"a7c8557432b3efb5ec12a423025fe83be9d8492bad352fdd57ab810f46e39660"}
Nov 02 15:05:35.162672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:35.162611    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2" podStartSLOduration=2.645615269 podStartE2EDuration="5.162595918s" totalImagesPullingTime="2.516980649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:05:35.161975019 +0000 UTC m=+541.599877160" watchObservedRunningTime="2026-11-02 15:05:35.162595918 +0000 UTC m=+541.600498059"
Nov 02 15:05:35.413875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:35.413796    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:36.700030 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:36.699990    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:36.713429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:36.713399    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:37.058044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:37.057964    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2"
Nov 02 15:05:41.414159 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:41.414114    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:41.426796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:41.426766    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:42.080890 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:42.080858    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2"
Nov 02 15:05:43.506578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.506541    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"]
Nov 02 15:05:43.510292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.510262    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.562869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.562836    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.563046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.562889    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da922da-c4c3-4c35-9edb-45f6bd072094-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.563046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.563023    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.563173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.563111    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-556sn\" (UniqueName: \"kubernetes.io/projected/1da922da-c4c3-4c35-9edb-45f6bd072094-kube-api-access-556sn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.563265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.563252    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.563307 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.563275    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.583296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.583260    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Nov 02 15:05:43.613387 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.613352    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"]
Nov 02 15:05:43.663738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663701    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.663940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663751    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da922da-c4c3-4c35-9edb-45f6bd072094-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.663940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663802    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.663940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663837    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-556sn\" (UniqueName: \"kubernetes.io/projected/1da922da-c4c3-4c35-9edb-45f6bd072094-kube-api-access-556sn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.663940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663909    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.663940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.663932    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.664272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.664178    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.664272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.664237    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.664394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.664333    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.666255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.666231    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1da922da-c4c3-4c35-9edb-45f6bd072094-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.666559 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.666538    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1da922da-c4c3-4c35-9edb-45f6bd072094-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.727435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.727401    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-556sn\" (UniqueName: \"kubernetes.io/projected/1da922da-c4c3-4c35-9edb-45f6bd072094-kube-api-access-556sn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7\" (UID: \"1da922da-c4c3-4c35-9edb-45f6bd072094\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:43.824118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:43.824006    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:44.202496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:44.202469    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"]
Nov 02 15:05:44.205215 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:44.205159    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da922da_c4c3_4c35_9edb_45f6bd072094.slice/crio-bba442854c664a3023c768a052b4650181055956c8796a4eeec00fde5c332847 WatchSource:0}: Error finding container bba442854c664a3023c768a052b4650181055956c8796a4eeec00fde5c332847: Status 404 returned error can't find the container with id bba442854c664a3023c768a052b4650181055956c8796a4eeec00fde5c332847
Nov 02 15:05:45.084158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:45.084066    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7" event={"ID":"1da922da-c4c3-4c35-9edb-45f6bd072094","Type":"ContainerStarted","Data":"7d422ccc08c1d2bbb6b010ba8afeee7085098c1375afb9e234256790581579cc"}
Nov 02 15:05:45.084158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:45.084117    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7" event={"ID":"1da922da-c4c3-4c35-9edb-45f6bd072094","Type":"ContainerStarted","Data":"bba442854c664a3023c768a052b4650181055956c8796a4eeec00fde5c332847"}
Nov 02 15:05:45.203629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:45.203570    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7" podStartSLOduration=1.599182699 podStartE2EDuration="2.203554027s" totalImagesPullingTime="604.371328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:05:45.201799019 +0000 UTC m=+551.639701161" watchObservedRunningTime="2026-11-02 15:05:45.203554027 +0000 UTC m=+551.641456168"
Nov 02 15:05:45.824296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:45.824254    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:48.532749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.532714    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"]
Nov 02 15:05:48.536251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.536229    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.588589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.588553    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6x9j8\""
Nov 02 15:05:48.601804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.601774    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.601893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.601808    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.601893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.601834    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.601995 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.601972    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkfqq\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-kube-api-access-nkfqq\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.602034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.602015    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.602066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.602042    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.602096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.602062    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.602141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.602129    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.602175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.602156    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92bb99de-1e3b-4f61-9cac-13b3c9496758-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.641610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.641544    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"]
Nov 02 15:05:48.703224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703173    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkfqq\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-kube-api-access-nkfqq\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703238    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703284    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703348    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703374    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703412    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92bb99de-1e3b-4f61-9cac-13b3c9496758-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703545    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703569    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.703709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703622    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.704043 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.703995    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.704183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.704088    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.704314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.704215    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.704453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.704430    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/92bb99de-1e3b-4f61-9cac-13b3c9496758-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.704540 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.704521    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.705852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.705834    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.706655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.706627    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.768262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.768225    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.769153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.769128    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkfqq\" (UniqueName: \"kubernetes.io/projected/92bb99de-1e3b-4f61-9cac-13b3c9496758-kube-api-access-nkfqq\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-vdl45\" (UID: \"92bb99de-1e3b-4f61-9cac-13b3c9496758\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:48.856523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:48.856429    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:49.025503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.025465    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"]
Nov 02 15:05:49.029113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.029093    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.081779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.081750    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Nov 02 15:05:49.089482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.089449    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"]
Nov 02 15:05:49.107968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.107892    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.107968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.107930    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mz5x\" (UniqueName: \"kubernetes.io/projected/0aa36d7b-d894-497a-8f79-3f68a36b0f88-kube-api-access-4mz5x\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.108148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.108029    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.108148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.108059    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.108148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.108098    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.108274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.108255    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209461    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209524    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209560    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209579    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mz5x\" (UniqueName: \"kubernetes.io/projected/0aa36d7b-d894-497a-8f79-3f68a36b0f88-kube-api-access-4mz5x\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209617    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.209702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.209649    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.210052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.210021    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.210052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.210043    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.210144 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.210070    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.211933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.211911    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0aa36d7b-d894-497a-8f79-3f68a36b0f88-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.212137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.212117    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0aa36d7b-d894-497a-8f79-3f68a36b0f88-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.270812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.270779    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mz5x\" (UniqueName: \"kubernetes.io/projected/0aa36d7b-d894-497a-8f79-3f68a36b0f88-kube-api-access-4mz5x\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-rqxwp\" (UID: \"0aa36d7b-d894-497a-8f79-3f68a36b0f88\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.318772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.318596    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"]
Nov 02 15:05:49.321629 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:49.321593    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92bb99de_1e3b_4f61_9cac_13b3c9496758.slice/crio-05bfabe84dcfd56cd2583a4d9b6cde73fabe8f7e236b2b461b933e1e147ca10d WatchSource:0}: Error finding container 05bfabe84dcfd56cd2583a4d9b6cde73fabe8f7e236b2b461b933e1e147ca10d: Status 404 returned error can't find the container with id 05bfabe84dcfd56cd2583a4d9b6cde73fabe8f7e236b2b461b933e1e147ca10d
Nov 02 15:05:49.342882 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.342843    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:49.720495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:49.720459    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"]
Nov 02 15:05:49.723496 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:49.723457    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa36d7b_d894_497a_8f79_3f68a36b0f88.slice/crio-921a930f689b33bd12a2676a60839d6f1d42884d2d0c83dd97f3dae62c5fae23 WatchSource:0}: Error finding container 921a930f689b33bd12a2676a60839d6f1d42884d2d0c83dd97f3dae62c5fae23: Status 404 returned error can't find the container with id 921a930f689b33bd12a2676a60839d6f1d42884d2d0c83dd97f3dae62c5fae23
Nov 02 15:05:50.106937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:50.106834    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp" event={"ID":"0aa36d7b-d894-497a-8f79-3f68a36b0f88","Type":"ContainerStarted","Data":"921a930f689b33bd12a2676a60839d6f1d42884d2d0c83dd97f3dae62c5fae23"}
Nov 02 15:05:50.108077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:50.108053    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" event={"ID":"92bb99de-1e3b-4f61-9cac-13b3c9496758","Type":"ContainerStarted","Data":"05bfabe84dcfd56cd2583a4d9b6cde73fabe8f7e236b2b461b933e1e147ca10d"}
Nov 02 15:05:51.114410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:51.114366    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp" event={"ID":"0aa36d7b-d894-497a-8f79-3f68a36b0f88","Type":"ContainerStarted","Data":"be4d586a99b9a41ba08b9dc2b76dd68e8d1f2ba0f87d712a03d5642f5ec67cba"}
Nov 02 15:05:51.233417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:51.233364    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp" podStartSLOduration=2.642491891 podStartE2EDuration="3.233349242s" totalImagesPullingTime="590.857351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:05:51.232270942 +0000 UTC m=+557.670173098" watchObservedRunningTime="2026-11-02 15:05:51.233349242 +0000 UTC m=+557.671251383"
Nov 02 15:05:51.343834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:51.343795    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:53.455344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:53.455310    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:05:53.455626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:53.455385    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:05:53.455626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:53.455417    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:05:53.824374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:53.824282    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:53.841444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:53.841408    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:54.133113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.133020    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" event={"ID":"92bb99de-1e3b-4f61-9cac-13b3c9496758","Type":"ContainerStarted","Data":"07386b8bad806c17640c6c0697a5e5290c539203248e8e1db04ec94491808b62"}
Nov 02 15:05:54.144783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.144751    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7"
Nov 02 15:05:54.265844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.265784    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podStartSLOduration=2.135269546 podStartE2EDuration="6.26576682s" totalImagesPullingTime="4.130497274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:05:54.263814079 +0000 UTC m=+560.701716220" watchObservedRunningTime="2026-11-02 15:05:54.26576682 +0000 UTC m=+560.703668960"
Nov 02 15:05:54.856914 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.856872    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:05:54.858406 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.858378    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:54.858546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:54.858440    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:05:55.857027 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:55.856987    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:55.857462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:55.857042    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:05:56.857122 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:56.857083    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:56.857530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:56.857161    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:05:57.857767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:57.857723    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:57.858278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:57.857785    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:05:58.806723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.806674    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"]
Nov 02 15:05:58.810538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.810516    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.851474 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.851442    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"]
Nov 02 15:05:58.856958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.856925    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:58.857098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.857001    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:05:58.898950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.898909    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 02 15:05:58.927665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.927626    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vjhv\" (UniqueName: \"kubernetes.io/projected/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-kube-api-access-8vjhv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.927796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.927694    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.927796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.927745    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.927875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.927836    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.927921 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.927887    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:58.928030 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:58.928011    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.028887 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.028818    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vjhv\" (UniqueName: \"kubernetes.io/projected/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-kube-api-access-8vjhv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.028938    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.028965    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.028992    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.029053    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.029138    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.029484    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.029506    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.029683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.029608    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.031584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.031561    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.031788 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.031771    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.097257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.097159    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vjhv\" (UniqueName: \"kubernetes.io/projected/f2ace023-874e-4aa4-a6c7-d91d3f83fb1c-kube-api-access-8vjhv\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-mdbl2\" (UID: \"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.123041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.123009    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:05:59.343333 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.343293    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:59.356100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.356011    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:05:59.526091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.526056    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"]
Nov 02 15:05:59.529764 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:05:59.529731    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ace023_874e_4aa4_a6c7_d91d3f83fb1c.slice/crio-a4926c65e02de413c4a0e8e05e581926e72c7e07282c1121ae25729b8271f658 WatchSource:0}: Error finding container a4926c65e02de413c4a0e8e05e581926e72c7e07282c1121ae25729b8271f658: Status 404 returned error can't find the container with id a4926c65e02de413c4a0e8e05e581926e72c7e07282c1121ae25729b8271f658
Nov 02 15:05:59.857385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.857347    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body=
Nov 02 15:05:59.857607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:05:59.857398    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused"
Nov 02 15:06:00.166567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:00.166534    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2" event={"ID":"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c","Type":"ContainerStarted","Data":"a4926c65e02de413c4a0e8e05e581926e72c7e07282c1121ae25729b8271f658"}
Nov 02 15:06:00.179120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:00.179087    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-rqxwp"
Nov 02 15:06:01.173527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:01.173480    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2" event={"ID":"f2ace023-874e-4aa4-a6c7-d91d3f83fb1c","Type":"ContainerStarted","Data":"15ed77ae377fed3c37ea21a4ca9d9057e0036755d674ebd8ecf41e038928165f"}
Nov 02 15:06:01.291391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:01.291331    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2" podStartSLOduration=2.673124247 podStartE2EDuration="3.291309186s" totalImagesPullingTime="618.184939ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:06:01.290744315 +0000 UTC m=+567.728646449" watchObservedRunningTime="2026-11-02 15:06:01.291309186 +0000 UTC m=+567.729211343"
Nov 02 15:06:01.898392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:01.898348    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-vdl45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:06:01.898624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:01.898423    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45" podUID="92bb99de-1e3b-4f61-9cac-13b3c9496758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:06:02.123649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:02.123604    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:06:02.593251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:02.593211    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:06:02.593838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:02.593649    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:06:02.594904 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:02.594881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-vdl45"
Nov 02 15:06:03.567010 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:06:03.566893    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:06:09.123763 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:09.123716    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:06:09.137239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:09.137204    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:06:09.223254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:09.223223    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-mdbl2"
Nov 02 15:06:13.937214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:13.937171    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:13.943774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:13.943744    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.000752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.000715    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Nov 02 15:06:14.050121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.050088    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-qg9kf\""
Nov 02 15:06:14.050357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.050338    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Nov 02 15:06:14.054842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.054817    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:14.084571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.084538    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.084731 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.084582    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8frmj\" (UniqueName: \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.185825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.185781    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.186031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.185841    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8frmj\" (UniqueName: \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.188184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.188133    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.270922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.270892    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8frmj\" (UniqueName: \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\") pod \"ai-gateway-controller-7958bf97d9-cwz7h\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") " pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.560142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.560051    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:14.941889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:14.941858    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:14.945112 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:06:14.945080    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc77af53_a56b_4fb2_91b8_da6c7f557000.slice/crio-846a67c5af5db6cc9d4dc7ad9d9d0e2bcc23e36c8a4047fc7f4de7b1ad6b13fa WatchSource:0}: Error finding container 846a67c5af5db6cc9d4dc7ad9d9d0e2bcc23e36c8a4047fc7f4de7b1ad6b13fa: Status 404 returned error can't find the container with id 846a67c5af5db6cc9d4dc7ad9d9d0e2bcc23e36c8a4047fc7f4de7b1ad6b13fa
Nov 02 15:06:15.239738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:15.239642    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" event={"ID":"cc77af53-a56b-4fb2-91b8-da6c7f557000","Type":"ContainerStarted","Data":"846a67c5af5db6cc9d4dc7ad9d9d0e2bcc23e36c8a4047fc7f4de7b1ad6b13fa"}
Nov 02 15:06:18.804231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.804140    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:06:18.807638 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.807615    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.911942 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.911908    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:06:18.931796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.931760    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.931959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.931803    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.931959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.931829    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.931946    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.931989    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jlpd\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.932175    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.932248    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.932280    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:18.932326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:18.932305    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033455    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033511    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033548    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033584    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033645    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033671    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033704    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033733    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.033935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033759    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jlpd\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.034034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.033927    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.034379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.034353    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.034550 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.034528    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.034597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.034580    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.034708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.034684    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.036173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.036148    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.036865 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.036840    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.095336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.095264    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.095508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.095487    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jlpd\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-f9td2\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.123298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.123259    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:19.563660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.563635    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:06:19.565949 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:06:19.565914    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14f04132_bc3d_495d_91e3_305a4fb8cf9c.slice/crio-03b667ab7327e5d02bb2d71e09bbca1fc427aea830314f80b71eef2e96a5f214 WatchSource:0}: Error finding container 03b667ab7327e5d02bb2d71e09bbca1fc427aea830314f80b71eef2e96a5f214: Status 404 returned error can't find the container with id 03b667ab7327e5d02bb2d71e09bbca1fc427aea830314f80b71eef2e96a5f214
Nov 02 15:06:19.568248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.568183    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:06:19.568318 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.568285    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:06:19.568367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:19.568316    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:06:20.265134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:20.265094    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" event={"ID":"cc77af53-a56b-4fb2-91b8-da6c7f557000","Type":"ContainerStarted","Data":"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"}
Nov 02 15:06:20.265612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:20.265168    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:20.266734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:20.266707    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" event={"ID":"14f04132-bc3d-495d-91e3-305a4fb8cf9c","Type":"ContainerStarted","Data":"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"}
Nov 02 15:06:20.266835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:20.266751    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" event={"ID":"14f04132-bc3d-495d-91e3-305a4fb8cf9c","Type":"ContainerStarted","Data":"03b667ab7327e5d02bb2d71e09bbca1fc427aea830314f80b71eef2e96a5f214"}
Nov 02 15:06:20.514997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:20.514935    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" podStartSLOduration=3.071335652 podStartE2EDuration="7.514917305s" totalImagesPullingTime="4.443581653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:06:20.393393983 +0000 UTC m=+586.831296125" watchObservedRunningTime="2026-11-02 15:06:20.514917305 +0000 UTC m=+586.952819448"
Nov 02 15:06:21.123659 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:21.123619    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:21.125247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:21.125215    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:21.125393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:21.125271    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:22.124173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:22.124132    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:22.124602 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:22.124211    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:23.124499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:23.124464    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:23.124983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:23.124529    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:24.124813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:24.124776    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:24.125233 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:24.124832    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:25.124238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:25.124180    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:25.124427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:25.124252    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:26.123950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:26.123910    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Nov 02 15:06:26.124363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:26.123962    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Nov 02 15:06:27.802918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:27.802865    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" podUID="6e7f87aa5e9c9e37d740f6ee1d873224" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:06:27.908499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:27.908467    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:27.908781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:27.908745    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:27.909599 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:27.909581    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:06:28.081325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:28.081223    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podStartSLOduration=10.081208771 podStartE2EDuration="10.081208771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:06:20.515024721 +0000 UTC m=+586.952926861" watchObservedRunningTime="2026-11-02 15:06:28.081208771 +0000 UTC m=+594.519110909"
Nov 02 15:06:31.297380 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.297342    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:31.655065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.654988    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"]
Nov 02 15:06:31.662323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.662297    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.683973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.683937    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34d7b14a-ff01-4807-99c4-14c4f9df1511-metrics-certs\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.684140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.684119    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69zhw\" (UniqueName: \"kubernetes.io/projected/34d7b14a-ff01-4807-99c4-14c4f9df1511-kube-api-access-69zhw\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.717511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.717479    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"]
Nov 02 15:06:31.785382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.785346    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69zhw\" (UniqueName: \"kubernetes.io/projected/34d7b14a-ff01-4807-99c4-14c4f9df1511-kube-api-access-69zhw\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.785594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.785406    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34d7b14a-ff01-4807-99c4-14c4f9df1511-metrics-certs\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.787813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.787788    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34d7b14a-ff01-4807-99c4-14c4f9df1511-metrics-certs\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.846615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.846574    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69zhw\" (UniqueName: \"kubernetes.io/projected/34d7b14a-ff01-4807-99c4-14c4f9df1511-kube-api-access-69zhw\") pod \"ai-gateway-controller-6f9b4d7758-wc49n\" (UID: \"34d7b14a-ff01-4807-99c4-14c4f9df1511\") " pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:31.976213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:31.976149    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:32.352301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:32.352277    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"]
Nov 02 15:06:32.354782 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:06:32.354755    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d7b14a_ff01_4807_99c4_14c4f9df1511.slice/crio-0ef5f07850ffae6371ae4b7ee3dd42ec6e1952073c3468982b1c5d8a9f282f66 WatchSource:0}: Error finding container 0ef5f07850ffae6371ae4b7ee3dd42ec6e1952073c3468982b1c5d8a9f282f66: Status 404 returned error can't find the container with id 0ef5f07850ffae6371ae4b7ee3dd42ec6e1952073c3468982b1c5d8a9f282f66
Nov 02 15:06:33.355243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.355181    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n" event={"ID":"34d7b14a-ff01-4807-99c4-14c4f9df1511","Type":"ContainerStarted","Data":"b2cb2b3f7c81a4802a6d0c0fc493de9d798ae89a99cf5cea7f433b312b25972c"}
Nov 02 15:06:33.355678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.355364    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:33.355678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.355388    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n" event={"ID":"34d7b14a-ff01-4807-99c4-14c4f9df1511","Type":"ContainerStarted","Data":"0ef5f07850ffae6371ae4b7ee3dd42ec6e1952073c3468982b1c5d8a9f282f66"}
Nov 02 15:06:33.473771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.473661    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n" podStartSLOduration=1.612854456 podStartE2EDuration="2.473645162s" totalImagesPullingTime="860.790706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:06:33.472640353 +0000 UTC m=+599.910542504" watchObservedRunningTime="2026-11-02 15:06:33.473645162 +0000 UTC m=+599.911547302"
Nov 02 15:06:33.564816 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:06:33.564779    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:06:33.745302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.745220    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:06:33.745484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:33.745467    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:06:44.361874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.361836    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6f9b4d7758-wc49n"
Nov 02 15:06:44.529708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.529670    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:44.530176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.530138    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" podUID="cc77af53-a56b-4fb2-91b8-da6c7f557000" containerName="manager" containerID="cri-o://cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239" gracePeriod=30
Nov 02 15:06:44.784768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.784743    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:44.915900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.915820    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\") pod \"cc77af53-a56b-4fb2-91b8-da6c7f557000\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") "
Nov 02 15:06:44.916048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.915900    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\" (UniqueName: \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\") pod \"cc77af53-a56b-4fb2-91b8-da6c7f557000\" (UID: \"cc77af53-a56b-4fb2-91b8-da6c7f557000\") "
Nov 02 15:06:44.918077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.918046    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs" pod "cc77af53-a56b-4fb2-91b8-da6c7f557000" (UID: "cc77af53-a56b-4fb2-91b8-da6c7f557000"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:06:44.918213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:44.918125    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj" pod "cc77af53-a56b-4fb2-91b8-da6c7f557000" (UID: "cc77af53-a56b-4fb2-91b8-da6c7f557000"). InnerVolumeSpecName "kube-api-access-8frmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:06:45.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.017560    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc77af53-a56b-4fb2-91b8-da6c7f557000-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:06:45.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.017593    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8frmj\" (UniqueName: \"kubernetes.io/projected/cc77af53-a56b-4fb2-91b8-da6c7f557000-kube-api-access-8frmj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:06:45.410957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.410917    4470 generic.go:356] "Generic (PLEG): container finished" podID="cc77af53-a56b-4fb2-91b8-da6c7f557000" containerID="cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239" exitCode=0
Nov 02 15:06:45.411461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.410960    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"
Nov 02 15:06:45.411461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.410990    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" event={"ID":"cc77af53-a56b-4fb2-91b8-da6c7f557000","Type":"ContainerDied","Data":"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"}
Nov 02 15:06:45.411461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.411070    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7958bf97d9-cwz7h" event={"ID":"cc77af53-a56b-4fb2-91b8-da6c7f557000","Type":"ContainerDied","Data":"846a67c5af5db6cc9d4dc7ad9d9d0e2bcc23e36c8a4047fc7f4de7b1ad6b13fa"}
Nov 02 15:06:45.411461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.411091    4470 scope.go:122] "RemoveContainer" containerID="cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"
Nov 02 15:06:45.420326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.420312    4470 scope.go:122] "RemoveContainer" containerID="cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"
Nov 02 15:06:45.420595 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:06:45.420571    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239\": container with ID starting with cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239 not found: ID does not exist" containerID="cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"
Nov 02 15:06:45.420646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.420606    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239"} err="failed to get container status \"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239\": rpc error: code = NotFound desc = could not find container \"cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239\": container with ID starting with cd636eb31ba189c8e8e25d9509eb67e5ebb1f8a888849131f5f1ed20db635239 not found: ID does not exist"
Nov 02 15:06:45.583198 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.583151    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:45.632314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.632278    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7958bf97d9-cwz7h"]
Nov 02 15:06:45.704589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:06:45.704557    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc77af53-a56b-4fb2-91b8-da6c7f557000" path="/var/lib/kubelet/pods/cc77af53-a56b-4fb2-91b8-da6c7f557000/volumes"
Nov 02 15:07:03.565383 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:03.565338    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:07:15.594273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.594235    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:07:15.595023 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.594663    4470 cpu_manager.go:414] "containerMap: removing container" podUID="cc77af53-a56b-4fb2-91b8-da6c7f557000" containerName="manager"
Nov 02 15:07:15.595023 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.594674    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc77af53-a56b-4fb2-91b8-da6c7f557000" containerName="manager"
Nov 02 15:07:15.595023 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.594745    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc77af53-a56b-4fb2-91b8-da6c7f557000" containerName="manager"
Nov 02 15:07:15.599202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.599166    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625141    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625230    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86qmq\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625265    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625376    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625463    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625505    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625528    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625670    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.625757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.625711    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.667261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.667226    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:07:15.727168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727137    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-db3931-openshift-default-dockercfg-w9t9c\""
Nov 02 15:07:15.727168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727144    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727226    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727297    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727349    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86qmq\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727373    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727397    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727432    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727460    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727492    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727760    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727787    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.727988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.727969    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.728293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.728037    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.728360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.728299    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.729609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.729579    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.729957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.729935    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.813007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.812974    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86qmq\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.823933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.823727    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:15.913807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:15.913705    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:16.256436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.256401    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:07:16.261068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.261046    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.313829 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.313802    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-42555d-openshift-default-dockercfg-4gl7w\""
Nov 02 15:07:16.326143 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.326108    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:07:16.334055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334025    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334069    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334158    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334322    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982nh\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334361    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334499    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334602 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334563    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334602 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334593    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.334709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.334641    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.405648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.403533    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:07:16.415072 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:16.415041    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15733bb9_1506_4ab3_b622_217c34be5091.slice/crio-08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3 WatchSource:0}: Error finding container 08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3: Status 404 returned error can't find the container with id 08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3
Nov 02 15:07:16.419654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.418506    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:16.419654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.418692    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:16.419654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.418750    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:16.435330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435281    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982nh\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435337    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435414    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435455    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435486    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435522    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435568    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435623    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.435931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.435650    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.437147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.436974    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.438613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.438528    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.438774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.438718    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.438903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.438876    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.439283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.439237    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.441889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.441863    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.442047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.442025    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.501627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.501596    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.516951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.516920    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-982nh\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.563813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.563779    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerStarted","Data":"1e9bdf4c44db504a844d430dc2789818088d47bf214ad6f6136490b4b8b4c817"}
Nov 02 15:07:16.564022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.563837    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerStarted","Data":"08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3"}
Nov 02 15:07:16.574165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.574138    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:16.691319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.691249    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podStartSLOduration=1.691224565 podStartE2EDuration="1.691224565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:16.688833193 +0000 UTC m=+643.126735361" watchObservedRunningTime="2026-11-02 15:07:16.691224565 +0000 UTC m=+643.129126708"
Nov 02 15:07:16.914665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.914571    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:16.918786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.918759    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:07:16.923782 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.923761    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:16.976824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.976789    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-019efe-openshift-default-dockercfg-9lzm6\""
Nov 02 15:07:16.985674 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:16.985647    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:07:17.015147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.015116    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:07:17.018728 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:17.018703    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00b64196_9cfa_42fb_8e9c_d6237ba80507.slice/crio-7b3978e3abf3b2c6045c9ba387914924d2e32ee0de58ffae6cba2ad90e00e467 WatchSource:0}: Error finding container 7b3978e3abf3b2c6045c9ba387914924d2e32ee0de58ffae6cba2ad90e00e467: Status 404 returned error can't find the container with id 7b3978e3abf3b2c6045c9ba387914924d2e32ee0de58ffae6cba2ad90e00e467
Nov 02 15:07:17.021309 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.021278    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.021403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.021345    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.021403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.021381    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.042806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.042779    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.042963 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.042832    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.042963 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.042915    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.042962    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.043031    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.043298    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bmnf\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.043359    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.043508    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.043613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.043576    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144389    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144442    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144465    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144488    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144513    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144558    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144579    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bmnf\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.144623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144616    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144637    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144882    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.145145    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.144899    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.145432    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.145568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.145549    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.148805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.148779    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.149021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.149001    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.213601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.213565    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.214530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.214492    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bmnf\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.254989 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.254958    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:17.329292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.329255    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:07:17.335654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.335623    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.345982    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346057    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346156    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346234    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346284    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346344    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85298\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346364    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346396    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.346530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.346436    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.398239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.398202    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-c18e79-openshift-default-dockercfg-2fs8l\""
Nov 02 15:07:17.408257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.408220    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:07:17.447284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447237    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447296    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85298\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447317    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447333    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447369    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447413    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447478    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447527    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447569    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.447884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447851    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.448013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.447966    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.448013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.448001    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.448141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.448054    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.448490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.448468    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.450178 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.450155    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.450298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.450221    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.508551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.508473    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85298\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.510266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.510246    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.576749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.576708    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerStarted","Data":"545af7fdc79274c2f0f8a2a414a497d2f459cc87e3f4d9f50a1539bc3454acf3"}
Nov 02 15:07:17.576950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.576770    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerStarted","Data":"7b3978e3abf3b2c6045c9ba387914924d2e32ee0de58ffae6cba2ad90e00e467"}
Nov 02 15:07:17.651118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.651088    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:17.690554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.690522    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:07:17.693891 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:17.693848    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31c3a95_47cd_4135_9009_2e8fc2320881.slice/crio-394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628 WatchSource:0}: Error finding container 394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628: Status 404 returned error can't find the container with id 394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628
Nov 02 15:07:17.697132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.696630    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.697132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.696735    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.697132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.696783    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:17.699775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.699723    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podStartSLOduration=1.699706229 podStartE2EDuration="1.699706229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:17.698076385 +0000 UTC m=+644.135978527" watchObservedRunningTime="2026-11-02 15:07:17.699706229 +0000 UTC m=+644.137608370"
Nov 02 15:07:17.919534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:17.919504    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:18.047723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.047562    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:07:18.050849 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:18.050808    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79d6e7a_b8f0_41bc_af9c_03b490c6ae19.slice/crio-34eada36b4e3777b113ec3882d0515acd224eae2f3d66669ba683068f75f2602 WatchSource:0}: Error finding container 34eada36b4e3777b113ec3882d0515acd224eae2f3d66669ba683068f75f2602: Status 404 returned error can't find the container with id 34eada36b4e3777b113ec3882d0515acd224eae2f3d66669ba683068f75f2602
Nov 02 15:07:18.053940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.053896    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:18.054106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.054009    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:18.054106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.054057    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:18.574508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.574475    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:18.580284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.580259    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:18.583331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.583292    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerStarted","Data":"deb8f743eac465840bb7e0a42851dee08c4da6fde1c6d43057cea23ff484e763"}
Nov 02 15:07:18.583483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.583351    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerStarted","Data":"394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628"}
Nov 02 15:07:18.584912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.584885    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerStarted","Data":"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"}
Nov 02 15:07:18.585015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.584930    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerStarted","Data":"34eada36b4e3777b113ec3882d0515acd224eae2f3d66669ba683068f75f2602"}
Nov 02 15:07:18.585408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.585377    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:18.585520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.585431    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:18.586056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.586034    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:07:18.586140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.586114    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:18.652175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.652140    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:18.828530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:18.828423    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podStartSLOduration=2.828407963 podStartE2EDuration="2.828407963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:18.827002283 +0000 UTC m=+645.264904438" watchObservedRunningTime="2026-11-02 15:07:18.828407963 +0000 UTC m=+645.266310104"
Nov 02 15:07:19.097913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.097807    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podStartSLOduration=2.097789109 podStartE2EDuration="2.097789109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:19.095630546 +0000 UTC m=+645.533532709" watchObservedRunningTime="2026-11-02 15:07:19.097789109 +0000 UTC m=+645.535691250"
Nov 02 15:07:19.255645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.255613    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:19.261293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.261262    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:19.590949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.590880    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:19.591932 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.591908    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:07:19.658080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:19.658046    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:20.594729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:20.594689    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:20.595654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:20.595632    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:07:25.745462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.745429    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:07:25.750419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.750397    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.809833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.809801    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:07:25.833349 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.833317    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-db3931-serving-cert\""
Nov 02 15:07:25.852765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.852735    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.852960 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.852809    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.852960 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.852893    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.853082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.852958    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pjqs\" (UniqueName: \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.954479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.954444    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.954479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.954484    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.954752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.954561    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pjqs\" (UniqueName: \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.954752 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:25.954579    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-serving-cert: secret "maas-api-e2e-worker-w5-db3931-serving-cert" not found
Nov 02 15:07:25.954752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.954638    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.954752 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:25.954657    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls podName:07ad61b5-e9b8-49c2-91a6-88ef072cd430 nodeName:}" failed. No retries permitted until 2026-11-02 15:07:26.454635428 +0000 UTC m=+652.892537550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls") pod "maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" (UID: "07ad61b5-e9b8-49c2-91a6-88ef072cd430") : secret "maas-api-e2e-worker-w5-db3931-serving-cert" not found
Nov 02 15:07:25.955232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.955182    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:25.956954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:25.956936    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:26.017144 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:26.017070    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pjqs\" (UniqueName: \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:26.460161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:26.460126    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:26.462733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:26.462707    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") pod \"maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:26.663353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:26.663318    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:27.040981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:27.040950    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:07:27.044059 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:27.044027    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ad61b5_e9b8_49c2_91a6_88ef072cd430.slice/crio-4026b3921e55a765dbf49f23e360ccdc2e07db018203b06a532b2024acb0a0d5 WatchSource:0}: Error finding container 4026b3921e55a765dbf49f23e360ccdc2e07db018203b06a532b2024acb0a0d5: Status 404 returned error can't find the container with id 4026b3921e55a765dbf49f23e360ccdc2e07db018203b06a532b2024acb0a0d5
Nov 02 15:07:27.045345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:27.045328    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:07:27.669139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:27.669096    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" event={"ID":"07ad61b5-e9b8-49c2-91a6-88ef072cd430","Type":"ContainerStarted","Data":"4026b3921e55a765dbf49f23e360ccdc2e07db018203b06a532b2024acb0a0d5"}
Nov 02 15:07:28.676041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:28.675986    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" event={"ID":"07ad61b5-e9b8-49c2-91a6-88ef072cd430","Type":"ContainerStarted","Data":"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"}
Nov 02 15:07:28.676502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:28.676112    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:28.793825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:28.793767    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" podStartSLOduration=2.840325229 podStartE2EDuration="3.793746255s" totalImagesPullingTime="953.421026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:28.792999422 +0000 UTC m=+655.230901564" watchObservedRunningTime="2026-11-02 15:07:28.793746255 +0000 UTC m=+655.231648395"
Nov 02 15:07:33.564329 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:33.564299    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:07:34.596294 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.596253    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:07:34.601256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.601234    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.609321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.609041    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkn7s\" (UniqueName: \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.609321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.609093    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.656237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.656210    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-019efe\""
Nov 02 15:07:34.658505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.658476    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:07:34.733472 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.733436    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-019efe-dockercfg-4frbg\""
Nov 02 15:07:34.736898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.736656    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkn7s\" (UniqueName: \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.736898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.736744    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.737913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.737884    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.738275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.738232    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:07:34.752539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.752512    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:07:34.756138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.756120    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:34.800611 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.800582    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkn7s\" (UniqueName: \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\") pod \"payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.814866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.814839    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:07:34.838402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.838357    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:34.838561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.838480    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb5vk\" (UniqueName: \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:34.915907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.915791    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:34.939384 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.939351    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:34.939521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.939413    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb5vk\" (UniqueName: \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:34.940140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:34.940118    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:35.002849 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.002777    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb5vk\" (UniqueName: \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\") pod \"payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:35.068724 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.068686    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:35.290379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.290353    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:07:35.292617 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:35.292584    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4306b86e_471a_4a5d_bdcd_3b679571fcb8.slice/crio-5afad91e2ea11e0b9c21032ab61f25f674c700b4f27f15696b63e80745da1a1b WatchSource:0}: Error finding container 5afad91e2ea11e0b9c21032ab61f25f674c700b4f27f15696b63e80745da1a1b: Status 404 returned error can't find the container with id 5afad91e2ea11e0b9c21032ab61f25f674c700b4f27f15696b63e80745da1a1b
Nov 02 15:07:35.449814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.449778    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:07:35.453161 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:35.453131    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4da8b94_0bcf_4788_b4a8_dde8ebcdede6.slice/crio-a5754711995ba5647e93968fc19d2c8f86a1f38e5dd3307de8424209dbefb5af WatchSource:0}: Error finding container a5754711995ba5647e93968fc19d2c8f86a1f38e5dd3307de8424209dbefb5af: Status 404 returned error can't find the container with id a5754711995ba5647e93968fc19d2c8f86a1f38e5dd3307de8424209dbefb5af
Nov 02 15:07:35.745530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.745493    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" event={"ID":"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6","Type":"ContainerStarted","Data":"a5754711995ba5647e93968fc19d2c8f86a1f38e5dd3307de8424209dbefb5af"}
Nov 02 15:07:35.746810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:35.746780    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" event={"ID":"4306b86e-471a-4a5d-bdcd-3b679571fcb8","Type":"ContainerStarted","Data":"5afad91e2ea11e0b9c21032ab61f25f674c700b4f27f15696b63e80745da1a1b"}
Nov 02 15:07:36.438280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.438247    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:07:36.442058 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.442024    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.546244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.546208    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-42555d-serving-cert\""
Nov 02 15:07:36.549510 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.549475    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:07:36.563645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.563589    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.563797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.563664    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qsnw\" (UniqueName: \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.563797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.563770    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.563931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.563912    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.664712 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.664667    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.664893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.664787    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.664893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.664855    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qsnw\" (UniqueName: \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.665010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.664912    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.665431 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:36.665130    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-serving-cert: secret "maas-api-e2e-worker-w3-42555d-serving-cert" not found
Nov 02 15:07:36.665431 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:36.665227    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls podName:e900aca8-347a-400f-8743-bb3fdd0988f0 nodeName:}" failed. No retries permitted until 2026-11-02 15:07:37.165184153 +0000 UTC m=+663.603086275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls") pod "maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" (UID: "e900aca8-347a-400f-8743-bb3fdd0988f0") : secret "maas-api-e2e-worker-w3-42555d-serving-cert" not found
Nov 02 15:07:36.665645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.665472    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.668555 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.668533    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:36.729971 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:36.729858    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qsnw\" (UniqueName: \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:37.182692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:37.182603    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:37.187268 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:37.187238    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") pod \"maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:37.359794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:37.359553    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:37.758164 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:37.758130    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:07:37.764923 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:37.764883    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode900aca8_347a_400f_8743_bb3fdd0988f0.slice/crio-a67beb48ce872c43424c0344deec9837f14d4f2aa10285d81682060228071f71 WatchSource:0}: Error finding container a67beb48ce872c43424c0344deec9837f14d4f2aa10285d81682060228071f71: Status 404 returned error can't find the container with id a67beb48ce872c43424c0344deec9837f14d4f2aa10285d81682060228071f71
Nov 02 15:07:37.777071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:37.777034    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" event={"ID":"e900aca8-347a-400f-8743-bb3fdd0988f0","Type":"ContainerStarted","Data":"a67beb48ce872c43424c0344deec9837f14d4f2aa10285d81682060228071f71"}
Nov 02 15:07:38.783906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.783870    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" event={"ID":"e900aca8-347a-400f-8743-bb3fdd0988f0","Type":"ContainerStarted","Data":"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"}
Nov 02 15:07:38.784352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.783978    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:38.785622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.785602    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" event={"ID":"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6","Type":"ContainerStarted","Data":"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"}
Nov 02 15:07:38.785696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.785648    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:38.787183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.787162    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" event={"ID":"4306b86e-471a-4a5d-bdcd-3b679571fcb8","Type":"ContainerStarted","Data":"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"}
Nov 02 15:07:38.787304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.787231    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:38.903851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:38.903762    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" podStartSLOduration=2.048861917 podStartE2EDuration="2.903747094s" totalImagesPullingTime="854.885177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:38.901732758 +0000 UTC m=+665.339634900" watchObservedRunningTime="2026-11-02 15:07:38.903747094 +0000 UTC m=+665.341649238"
Nov 02 15:07:39.018464 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:39.018390    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" podStartSLOduration=2.126377158 podStartE2EDuration="5.018373353s" totalImagesPullingTime="2.891996195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:39.017544948 +0000 UTC m=+665.455447089" watchObservedRunningTime="2026-11-02 15:07:39.018373353 +0000 UTC m=+665.456275495"
Nov 02 15:07:39.137137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:39.137065    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" podStartSLOduration=2.038952804 podStartE2EDuration="5.137047007s" totalImagesPullingTime="3.098094203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:39.134620002 +0000 UTC m=+665.572522156" watchObservedRunningTime="2026-11-02 15:07:39.137047007 +0000 UTC m=+665.574949147"
Nov 02 15:07:42.103597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.103554    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:07:42.108303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.108273    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.215930 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.215900    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-c18e79\""
Nov 02 15:07:42.216279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.216249    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-c18e79-dockercfg-ptj9h\""
Nov 02 15:07:42.218605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.218582    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:07:42.264214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.264150    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5n5q\" (UniqueName: \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.264400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.264323    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.365364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.365267    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5n5q\" (UniqueName: \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.365364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.365360    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.366069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.366041    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.426522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.426485    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5n5q\" (UniqueName: \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\") pod \"payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:42.724440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:42.724404    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:43.103179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:43.103149    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:07:43.105707 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:07:43.105681    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbd675a_602a_43f9_a364_42735927789e.slice/crio-c5626cc4f9071dceeef3922b22b97a8e97b946e025050df066b80cc88fd29ff0 WatchSource:0}: Error finding container c5626cc4f9071dceeef3922b22b97a8e97b946e025050df066b80cc88fd29ff0: Status 404 returned error can't find the container with id c5626cc4f9071dceeef3922b22b97a8e97b946e025050df066b80cc88fd29ff0
Nov 02 15:07:43.843223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:43.843161    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" event={"ID":"8fbd675a-602a-43f9-a364-42735927789e","Type":"ContainerStarted","Data":"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"}
Nov 02 15:07:43.843223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:43.843232    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" event={"ID":"8fbd675a-602a-43f9-a364-42735927789e","Type":"ContainerStarted","Data":"c5626cc4f9071dceeef3922b22b97a8e97b946e025050df066b80cc88fd29ff0"}
Nov 02 15:07:43.843523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:43.843267    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:43.961974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:43.961920    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" podStartSLOduration=1.9619048700000001 podStartE2EDuration="1.96190487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:43.959654656 +0000 UTC m=+670.397556800" watchObservedRunningTime="2026-11-02 15:07:43.96190487 +0000 UTC m=+670.399807011"
Nov 02 15:07:44.797958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:44.797928    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:07:47.147944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.147867    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:07:47.151992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.151971    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.204783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.204756    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-de35cd-serving-cert\""
Nov 02 15:07:47.255956 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.255926    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:07:47.339727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.339696    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zc4h\" (UniqueName: \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.339937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.339912    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.339995 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.339969    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.340035 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.340003    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.440864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.440825    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zc4h\" (UniqueName: \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.441053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.440992    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.441053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.441033    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.441174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.441061    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.441315 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:47.441262    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-serving-cert: secret "maas-api-e2e-worker-w2-de35cd-serving-cert" not found
Nov 02 15:07:47.441374 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:07:47.441332    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls podName:2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c nodeName:}" failed. No retries permitted until 2026-11-02 15:07:47.941310679 +0000 UTC m=+674.379212813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls") pod "maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" (UID: "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c") : secret "maas-api-e2e-worker-w2-de35cd-serving-cert" not found
Nov 02 15:07:47.442294 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.442262    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.444609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.444588    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.502985 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.502953    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zc4h\" (UniqueName: \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.945907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.945863    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:47.948377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:47.948356    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") pod \"maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:48.064952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:48.064918    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:48.490849 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:48.490741    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:07:48.895506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:48.895411    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" event={"ID":"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c","Type":"ContainerStarted","Data":"41aba0d2cbd712ee6cad19653065b93e316c2fc3e877b7df16e58ac3e9337f7e"}
Nov 02 15:07:49.796029 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:49.795996    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:07:49.796620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:49.796410    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:07:49.904157 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:49.904106    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" event={"ID":"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c","Type":"ContainerStarted","Data":"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"}
Nov 02 15:07:49.904365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:49.904254    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:50.033515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:50.033445    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" podStartSLOduration=2.196993354 podStartE2EDuration="3.033423967s" totalImagesPullingTime="836.430613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:07:50.030657685 +0000 UTC m=+676.468559827" watchObservedRunningTime="2026-11-02 15:07:50.033423967 +0000 UTC m=+676.471326108"
Nov 02 15:07:52.921408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.921374    4470 generic.go:356] "Generic (PLEG): container finished" podID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerID="545af7fdc79274c2f0f8a2a414a497d2f459cc87e3f4d9f50a1539bc3454acf3" exitCode=0
Nov 02 15:07:52.921863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.921449    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerDied","Data":"545af7fdc79274c2f0f8a2a414a497d2f459cc87e3f4d9f50a1539bc3454acf3"}
Nov 02 15:07:52.921863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.921846    4470 scope.go:122] "RemoveContainer" containerID="545af7fdc79274c2f0f8a2a414a497d2f459cc87e3f4d9f50a1539bc3454acf3"
Nov 02 15:07:52.923172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.923138    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:52.923272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.923223    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:52.923272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:52.923258    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:07:53.929136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:53.929095    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerStarted","Data":"9678ebe81e126d275900111996a37133d7a68922f62765b3294971edbbbb5eed"}
Nov 02 15:07:54.580651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:54.580615    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:07:54.581033 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:54.581004    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:54.581240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:54.581213    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:07:54.851719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:54.851644    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:07:55.582240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:55.576601    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:55.582240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:55.576673    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:07:55.914901 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:55.914823    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:07:56.589100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:56.588624    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:56.589100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:56.588703    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:07:57.589216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:57.579590    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:57.589216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:57.579659    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:07:58.590215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:58.583610    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:58.590215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:58.583675    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:07:59.574554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:59.574517    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:07:59.574753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:07:59.574593    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:08:00.579394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:00.579361    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:08:00.579820 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:00.579773    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:08:00.580825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:00.580798    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:08:03.565632 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:08:03.565596    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:08:08.429817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.429780    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:08:08.437181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.437157    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.467838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.467784    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.468014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.467936    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2dz\" (UniqueName: \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.468014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.467980    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.468014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.467998    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.541176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.541140    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-6dc69b-serving-cert\""
Nov 02 15:08:08.544001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.543969    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:08:08.569026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.568988    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.569255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.569083    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2dz\" (UniqueName: \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.569255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.569139    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.569255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.569166    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.591513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.569864    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.591513 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:08:08.570266    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-serving-cert: secret "maas-api-e2e-worker-w4-6dc69b-serving-cert" not found
Nov 02 15:08:08.591513 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:08:08.570331    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls podName:7ac4b109-4242-4ef3-bbe1-ca757eb493d5 nodeName:}" failed. No retries permitted until 2026-11-02 15:08:09.070310498 +0000 UTC m=+695.508212619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls") pod "maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" (UID: "7ac4b109-4242-4ef3-bbe1-ca757eb493d5") : secret "maas-api-e2e-worker-w4-6dc69b-serving-cert" not found
Nov 02 15:08:08.603544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.603505    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:08.631302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:08.631269    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2dz\" (UniqueName: \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:09.107095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:09.107047    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:09.109975 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:09.109916    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") pod \"maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:09.350859 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:09.350829    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:09.736671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:09.736637    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:08:09.738744 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:08:09.738683    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac4b109_4242_4ef3_bbe1_ca757eb493d5.slice/crio-f2f3d4b8639f287e3158ad6f4744f1284f73ac8834b2b60665eed0a30c6537c2 WatchSource:0}: Error finding container f2f3d4b8639f287e3158ad6f4744f1284f73ac8834b2b60665eed0a30c6537c2: Status 404 returned error can't find the container with id f2f3d4b8639f287e3158ad6f4744f1284f73ac8834b2b60665eed0a30c6537c2
Nov 02 15:08:10.064785 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:10.064681    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" event={"ID":"7ac4b109-4242-4ef3-bbe1-ca757eb493d5","Type":"ContainerStarted","Data":"f2f3d4b8639f287e3158ad6f4744f1284f73ac8834b2b60665eed0a30c6537c2"}
Nov 02 15:08:11.073383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:11.073343    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" event={"ID":"7ac4b109-4242-4ef3-bbe1-ca757eb493d5","Type":"ContainerStarted","Data":"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"}
Nov 02 15:08:11.073906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:11.073429    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:11.197083 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:11.197021    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" podStartSLOduration=2.363516804 podStartE2EDuration="3.1970032s" totalImagesPullingTime="833.486396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:08:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:08:11.195296158 +0000 UTC m=+697.633198301" watchObservedRunningTime="2026-11-02 15:08:11.1970032 +0000 UTC m=+697.634905376"
Nov 02 15:08:13.086547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.086455    4470 generic.go:356] "Generic (PLEG): container finished" podID="15733bb9-1506-4ab3-b622-217c34be5091" containerID="1e9bdf4c44db504a844d430dc2789818088d47bf214ad6f6136490b4b8b4c817" exitCode=0
Nov 02 15:08:13.086547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.086530    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerDied","Data":"1e9bdf4c44db504a844d430dc2789818088d47bf214ad6f6136490b4b8b4c817"}
Nov 02 15:08:13.087080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.086971    4470 scope.go:122] "RemoveContainer" containerID="1e9bdf4c44db504a844d430dc2789818088d47bf214ad6f6136490b4b8b4c817"
Nov 02 15:08:13.088346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088326    4470 generic.go:356] "Generic (PLEG): container finished" podID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerID="deb8f743eac465840bb7e0a42851dee08c4da6fde1c6d43057cea23ff484e763" exitCode=0
Nov 02 15:08:13.088421 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088384    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerDied","Data":"deb8f743eac465840bb7e0a42851dee08c4da6fde1c6d43057cea23ff484e763"}
Nov 02 15:08:13.088421 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088395    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:13.088492 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088464    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:13.088525 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088505    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:13.088701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.088688    4470 scope.go:122] "RemoveContainer" containerID="deb8f743eac465840bb7e0a42851dee08c4da6fde1c6d43057cea23ff484e763"
Nov 02 15:08:13.089922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.089891    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:13.090026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.089953    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:13.090026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:13.089986    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:14.143254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.143211    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerStarted","Data":"af23046874219a126da818ddb3143eba7c7ab4bccc9435c4770af511f886b989"}
Nov 02 15:08:14.145255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.145229    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerStarted","Data":"22ae92078e07bfc3def4e6d735d80b56f8ce6ba37e216e3a63851fa1dd42ed96"}
Nov 02 15:08:14.255172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.255140    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:08:14.255695 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.255668    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:14.255818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.255725    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:14.915686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.915645    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 15:08:14.919513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.919270    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 15:08:14.919513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:14.916240    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:08:15.255988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:15.255954    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:15.256453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:15.256009    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:15.922220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:15.920477    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 15:08:15.922220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:15.920539    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 15:08:16.256232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:16.256172    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:16.256745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:16.256263    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:16.917215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:16.916790    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 15:08:16.917215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:16.916870    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 15:08:17.121215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:17.120470    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:08:17.255999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:17.255959    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:17.256218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:17.256012    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:17.919918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:17.919607    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 15:08:17.919918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:17.919675    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 15:08:18.256155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:18.256112    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:18.256368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:18.256182    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:18.590405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:18.590304    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:08:18.914736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:18.914631    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 02 15:08:18.915012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:18.914983    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 02 15:08:19.255979 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:19.255946    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 02 15:08:19.256458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:19.256000    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 02 15:08:19.591170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:19.591089    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:08:20.036552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:20.036518    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:08:20.038370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:20.038342    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:08:20.260328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:20.260292    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:08:20.261290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:20.261264    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:08:33.565760 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:08:33.565724    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:08:36.636401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:36.636358    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:08:36.636899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:36.636431    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:08:39.345470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.345371    4470 generic.go:356] "Generic (PLEG): container finished" podID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerID="1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff" exitCode=0
Nov 02 15:08:39.345470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.345448    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerDied","Data":"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"}
Nov 02 15:08:39.346068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.345947    4470 scope.go:122] "RemoveContainer" containerID="1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"
Nov 02 15:08:39.347876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.347817    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:39.347992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.347971    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:39.348073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:39.348048    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:08:40.352760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:40.352718    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerStarted","Data":"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"}
Nov 02 15:08:40.652109 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:40.652011    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:08:40.652298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:40.652267    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 15:08:40.652345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:40.652320    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 15:08:41.652183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:41.652142    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 15:08:41.652643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:41.652242    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 15:08:42.651699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:42.651656    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 15:08:42.651895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:42.651718    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 15:08:43.651871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:43.651832    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 15:08:43.652316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:43.651891    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 15:08:44.653570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:44.651917    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 02 15:08:44.653570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:44.651993    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 02 15:08:45.657282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:45.657246    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:08:45.657710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:45.657554    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:08:45.658317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:08:45.658294    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:09:03.564868 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:09:03.564829    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:09:33.565611 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:09:33.565570    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:10:02.074495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.074457    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:10:02.077358 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.077337    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.135815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.135773    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-42555d\"/\"openshift-service-ca.crt\""
Nov 02 15:10:02.170141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170110    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.170322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170148    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.170322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170182    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.170322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170310    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7297\" (UniqueName: \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.170438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170402    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.170480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.170436    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.182409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.182380    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-42555d\"/\"kube-root-ca.crt\""
Nov 02 15:10:02.185003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.184982    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:10:02.185326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.185305    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-42555d\"/\"premium-w3-42555d-kserve-self-signed-certs\""
Nov 02 15:10:02.185412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.185326    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-42555d\"/\"default-dockercfg-96rjb\""
Nov 02 15:10:02.276867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.276819    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.277766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.277738    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7297\" (UniqueName: \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.278415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.278384    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.278590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.278570    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.278868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.278848    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.279009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.278994    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.279516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.279492    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.279894 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.279876    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.280269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.280247    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.287537 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.287509    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.304315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.296108    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.339658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.339577    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7297\" (UniqueName: \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\") pod \"premium-w3-42555d-kserve-f8b5c7668-r6jhz\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") " pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.388583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.388547    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:02.782002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.781941    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:10:02.784488 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:10:02.784459    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34303c70_a1e0_496c_838c_60cc45cdccea.slice/crio-aec2a7df34e9bbf4df6ce75eab59ad4aa6ec0cfad12c90ffa056200a94e48b8a WatchSource:0}: Error finding container aec2a7df34e9bbf4df6ce75eab59ad4aa6ec0cfad12c90ffa056200a94e48b8a: Status 404 returned error can't find the container with id aec2a7df34e9bbf4df6ce75eab59ad4aa6ec0cfad12c90ffa056200a94e48b8a
Nov 02 15:10:02.814560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:02.814528    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" event={"ID":"34303c70-a1e0-496c-838c-60cc45cdccea","Type":"ContainerStarted","Data":"aec2a7df34e9bbf4df6ce75eab59ad4aa6ec0cfad12c90ffa056200a94e48b8a"}
Nov 02 15:10:03.564481 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:10:03.564443    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:10:03.821631 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:03.821536    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" event={"ID":"34303c70-a1e0-496c-838c-60cc45cdccea","Type":"ContainerStarted","Data":"7055eb4152c677c5dd782153dc76fa36dce9c02b2d1be7c62c6c28015fe5dc1f"}
Nov 02 15:10:03.939066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:03.939007    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podStartSLOduration=1.938987993 podStartE2EDuration="1.938987993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:10:03.937750491 +0000 UTC m=+810.375652633" watchObservedRunningTime="2026-11-02 15:10:03.938987993 +0000 UTC m=+810.376890135"
Nov 02 15:10:04.388687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:04.388648    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:12.389363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:12.389317    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:12.416558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:12.416230    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:12.881973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:12.881943    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:10:33.564922 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:10:33.564881    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:10:36.633710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.633664    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:10:36.637815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.637792    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.739260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.739229    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-019efe\"/\"openshift-service-ca.crt\""
Nov 02 15:10:36.739260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.739256    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-019efe\"/\"default-dockercfg-2g9cz\""
Nov 02 15:10:36.739486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.739283    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-019efe\"/\"kube-root-ca.crt\""
Nov 02 15:10:36.739486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.739329    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-019efe\"/\"premium-w1-019efe-kserve-self-signed-certs\""
Nov 02 15:10:36.741844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.741820    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:10:36.792361 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792320    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sx2t\" (UniqueName: \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.792846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792702    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.792846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792777    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.792846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792807    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.792846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792847    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.793180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.792953    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.893798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893689    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.893798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893779    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sx2t\" (UniqueName: \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893853    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893906    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893934    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.893969    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.894467    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.894755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.894734    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.895068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.895044    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.896670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.896643    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.897147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.897125    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:36.955726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:36.955693    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sx2t\" (UniqueName: \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\") pod \"premium-w1-019efe-kserve-d4df5657-96ll2\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") " pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:37.250558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:37.250519    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:37.628795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:37.628560    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:10:37.631145 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:10:37.631114    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod297573e4_27ee_460d_b847_6ed4f8c45c6b.slice/crio-30bab4bd32c98dd3681e258dd03b8a5e44a35687b8e6896e43613a5c3bc7fcfe WatchSource:0}: Error finding container 30bab4bd32c98dd3681e258dd03b8a5e44a35687b8e6896e43613a5c3bc7fcfe: Status 404 returned error can't find the container with id 30bab4bd32c98dd3681e258dd03b8a5e44a35687b8e6896e43613a5c3bc7fcfe
Nov 02 15:10:38.043889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:38.043849    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" event={"ID":"297573e4-27ee-460d-b847-6ed4f8c45c6b","Type":"ContainerStarted","Data":"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"}
Nov 02 15:10:38.044428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:38.043903    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" event={"ID":"297573e4-27ee-460d-b847-6ed4f8c45c6b","Type":"ContainerStarted","Data":"30bab4bd32c98dd3681e258dd03b8a5e44a35687b8e6896e43613a5c3bc7fcfe"}
Nov 02 15:10:38.161787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:38.161731    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podStartSLOduration=2.161715226 podStartE2EDuration="2.161715226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:10:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:10:38.160178362 +0000 UTC m=+844.598080504" watchObservedRunningTime="2026-11-02 15:10:38.161715226 +0000 UTC m=+844.599617425"
Nov 02 15:10:38.251056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:38.251027    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:47.251651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:47.251560    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:47.263853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:47.263821    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:10:48.109314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:10:48.109282    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:11:03.566595 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:11:03.566551    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:11:33.565080 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:11:33.565033    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:11:33.665179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.665146    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:11:33.669087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.669068    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.721580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.721553    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-6dc69b-openshift-default-dockercfg-pbd2c\""
Nov 02 15:11:33.745061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745030    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745069    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745091    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745261    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745305    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745385    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745438    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745485    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.745567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.745550    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsl8d\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.777278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.777246    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:11:33.795358 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.795322    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:11:33.798012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.797988    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:11:33.846590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846509    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.846590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846572    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.846817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846616    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.846817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846685    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.846817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846721    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.846817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846799    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsl8d\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846840    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846880    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.846916    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.847021    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.847055    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.847074    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.847331    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.847535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.847509    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.848990 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.848962    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.849331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.849311    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.908474 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.908441    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.908735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.908711    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsl8d\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:33.982001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:33.981963    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:34.415781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:34.415754    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:11:34.418159 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:11:34.418128    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf348580c_2b74_46bf_8890_e86cfbbc0c19.slice/crio-473c1133533b58459f04020b7c8d291c25da6a9d87880fe7951a98d1fc7ded63 WatchSource:0}: Error finding container 473c1133533b58459f04020b7c8d291c25da6a9d87880fe7951a98d1fc7ded63: Status 404 returned error can't find the container with id 473c1133533b58459f04020b7c8d291c25da6a9d87880fe7951a98d1fc7ded63
Nov 02 15:11:34.420805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:34.420770    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:11:34.420901 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:34.420839    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:11:34.420901 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:34.420866    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:11:35.367754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.367712    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" event={"ID":"f348580c-2b74-46bf-8890-e86cfbbc0c19","Type":"ContainerStarted","Data":"5496d447f6ae23f9d899b11a7081ff9c5064d3743698d94cf1ac2034f3d9869c"}
Nov 02 15:11:35.368165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.367770    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" event={"ID":"f348580c-2b74-46bf-8890-e86cfbbc0c19","Type":"ContainerStarted","Data":"473c1133533b58459f04020b7c8d291c25da6a9d87880fe7951a98d1fc7ded63"}
Nov 02 15:11:35.497375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.497319    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podStartSLOduration=2.497303907 podStartE2EDuration="2.497303907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:11:35.494840087 +0000 UTC m=+901.932742230" watchObservedRunningTime="2026-11-02 15:11:35.497303907 +0000 UTC m=+901.935206044"
Nov 02 15:11:35.991621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.991585    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:35.994121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.994085    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 15:11:35.994353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:35.994313    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 15:11:36.991212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:36.990645    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 15:11:36.991212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:36.990716    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 15:11:37.988684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:37.988641    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 15:11:37.989031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:37.988997    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 15:11:38.990016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:38.989975    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 15:11:38.990673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:38.990636    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 15:11:39.992678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:39.992629    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 02 15:11:39.993306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:39.993270    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 02 15:11:41.211972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:41.211936    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:41.213160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:41.213126    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:11:41.214325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:11:41.214301    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:12:03.567148 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:12:03.567108    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:12:21.678767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.678662    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:12:21.683180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.683156    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.802584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.802550    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:12:21.852596 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852557    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.852791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852603    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.852791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852738    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.852791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852767    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.852895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852791    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.852947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.852928    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tfcd\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.853047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.853028    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.853099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.853087    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.853211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.853172    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954070    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954210    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954237    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954265    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954287    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954318    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954417    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tfcd\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954474    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954538    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954561    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.954836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954800    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.955004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954971    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.955004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.954992    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.955226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.955203    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.957051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.957017    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:21.957183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:21.957099    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:22.018653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.018612    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tfcd\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:22.036423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.036388    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-mzmxn\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:22.299097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.298997    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:22.736553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.736522    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:12:22.738921 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:12:22.738893    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5792826_79f4_4ceb_816d_6d717cb4f970.slice/crio-ebb972750d7b2227fe69585c25e2f245aff85e24a2fef8331ebcbc17b1870445 WatchSource:0}: Error finding container ebb972750d7b2227fe69585c25e2f245aff85e24a2fef8331ebcbc17b1870445: Status 404 returned error can't find the container with id ebb972750d7b2227fe69585c25e2f245aff85e24a2fef8331ebcbc17b1870445
Nov 02 15:12:22.741436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.741402    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:22.741507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.741494    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:22.741544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:22.741529    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:23.748348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:23.748313    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" event={"ID":"a5792826-79f4-4ceb-816d-6d717cb4f970","Type":"ContainerStarted","Data":"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"}
Nov 02 15:12:23.748935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:23.748910    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" event={"ID":"a5792826-79f4-4ceb-816d-6d717cb4f970","Type":"ContainerStarted","Data":"ebb972750d7b2227fe69585c25e2f245aff85e24a2fef8331ebcbc17b1870445"}
Nov 02 15:12:23.917572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:23.917513    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podStartSLOduration=2.917496111 podStartE2EDuration="2.917496111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:12:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:12:23.915874628 +0000 UTC m=+950.353776770" watchObservedRunningTime="2026-11-02 15:12:23.917496111 +0000 UTC m=+950.355398251"
Nov 02 15:12:24.305294 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:24.305262    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:24.306502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:24.306474    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:24.306709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:24.306684    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:25.301215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:25.300711    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:25.301215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:25.300774    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:26.299731 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:26.299686    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:26.299920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:26.299755    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:27.299583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:27.299542    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:27.300085 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:27.299597    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:28.300374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:28.300284    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:28.300374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:28.300348    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:29.299664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:29.299625    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:29.299847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:29.299697    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:30.300300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:30.300258    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 02 15:12:30.300701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:30.300315    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 02 15:12:31.352414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:31.352374    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:31.352933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:31.352748    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:31.353816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:31.353791    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:12:33.564767 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:12:33.564734    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:12:47.901763 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.901726    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:12:47.906856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.906838    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.943975 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.943926    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.944429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.944392    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvrv5\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946088    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946164    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946240    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946286    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946327    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946389    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.946437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.946434    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:47.967118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:47.967066    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-de35cd-openshift-default-dockercfg-wdrm7\""
Nov 02 15:12:48.021991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.021953    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:12:48.047850 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.047815    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.047850 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.047865    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.047932    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.047995    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvrv5\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048027    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048088    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048138    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048181    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048242    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.048789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.048658    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.049124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.049094    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.049488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.049457    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.049730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.049707    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.050166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.050140    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.052479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.052454    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.058172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.058146    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.110409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.110373    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.110581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.110484    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvrv5\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.236209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.231462    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:48.302316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.302280    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:12:48.308054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.307551    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.353915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.353866    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.353926    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354028    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354133    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354234    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354297    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354336    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl7d5\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354360    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.354435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.354403    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.369317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.369283    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455643    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455699    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455729    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl7d5\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455763    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455784    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455884    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455920    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.455939    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.456212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.456001    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.457425 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.457352    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.459530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.459467    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.460302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.460166    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.460777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.460714    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.461410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.461390    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.464693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.464590    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.465513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.465456    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.518176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.518094    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.518362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.518281    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl7d5\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.634214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.631441    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:48.676360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.676319    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:12:48.680874 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:12:48.680840    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb5fa8dc_6aea_427b_8cb7_cd39b8f4e877.slice/crio-657e78ebc40a6aa3b7967f8c2fbe9bbd8bafdcaafc5def6a33487631481fbae9 WatchSource:0}: Error finding container 657e78ebc40a6aa3b7967f8c2fbe9bbd8bafdcaafc5def6a33487631481fbae9: Status 404 returned error can't find the container with id 657e78ebc40a6aa3b7967f8c2fbe9bbd8bafdcaafc5def6a33487631481fbae9
Nov 02 15:12:48.684219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.684164    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:48.684320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.684251    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:48.684320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.684279    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:48.960773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.959294    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" event={"ID":"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877","Type":"ContainerStarted","Data":"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"}
Nov 02 15:12:48.960773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:48.959368    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" event={"ID":"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877","Type":"ContainerStarted","Data":"657e78ebc40a6aa3b7967f8c2fbe9bbd8bafdcaafc5def6a33487631481fbae9"}
Nov 02 15:12:49.090030 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.089965    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podStartSLOduration=2.089941769 podStartE2EDuration="2.089941769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:12:49.084010143 +0000 UTC m=+975.521912288" watchObservedRunningTime="2026-11-02 15:12:49.089941769 +0000 UTC m=+975.527843910"
Nov 02 15:12:49.131517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.131462    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:12:49.163046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.163002    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:49.163219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.163087    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:49.163219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.163118    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:12:49.234110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.234067    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:49.235519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.235493    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:49.235742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:49.235702    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:50.031452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.031252    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" event={"ID":"45f12615-7aa9-4452-81eb-d7122a681047","Type":"ContainerStarted","Data":"0b2e054a6e35f55d8758645cdd3e6b1a5b3adc243070918e7be800e72e067ff9"}
Nov 02 15:12:50.031452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.031361    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" event={"ID":"45f12615-7aa9-4452-81eb-d7122a681047","Type":"ContainerStarted","Data":"88833f3b9ba5e10e8f6638cd1083aa6409dc09643a38ee63ea6b484b5682f032"}
Nov 02 15:12:50.167308 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.167257    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podStartSLOduration=2.167242271 podStartE2EDuration="2.167242271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:12:50.165027222 +0000 UTC m=+976.602929364" watchObservedRunningTime="2026-11-02 15:12:50.167242271 +0000 UTC m=+976.605144412"
Nov 02 15:12:50.233425 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.233384    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:50.233612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.233443    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:50.632673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.632639    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:50.633970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.633946    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:50.634131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:50.634111    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:51.233138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:51.233100    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:51.233563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:51.233164    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:51.633003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:51.632909    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:51.633003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:51.632982    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:52.232987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:52.232945    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:52.233265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:52.233001    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:52.632654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:52.632566    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:52.632654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:52.632623    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:53.233398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:53.233363    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:53.233807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:53.233424    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:53.632787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:53.632703    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:53.632787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:53.632760    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:54.232804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:54.232766    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:54.233020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:54.232828    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:54.632738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:54.632637    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:54.632738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:54.632699    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:55.233293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:55.233255    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 02 15:12:55.233485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:55.233310    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 02 15:12:55.633042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:55.632933    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 02 15:12:55.633555 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:55.633023    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 02 15:12:56.237161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.237121    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:56.237463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.237430    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:56.238150 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.238131    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:12:56.636889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.636809    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:56.637327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.637224    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:12:56.638245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:12:56.638221    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:13:03.565155 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:13:03.565105    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:13:03.731257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.731208    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:13:03.741376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.741340    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.797134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.797096    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:13:03.821833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.821744    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtwwf\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.821833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.821803    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.821833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.821830    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822079 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822049    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822095    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822149    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822307    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822365    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.822502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.822483    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.843095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.843067    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ba49d0-openshift-default-dockercfg-66tsc\""
Nov 02 15:13:03.923980 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.923943    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.923993    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924026    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924051    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtwwf\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924083    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924108    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924231    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924265    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924285    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924805    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924859    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924859    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.924946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.924886    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.925301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.925272    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.926635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.926612    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.926764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.926744    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.985204 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.985157    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtwwf\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:03.985365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:03.985159    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:04.055764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:04.055720    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:04.494774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:04.494746    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:13:04.500037 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:13:04.500009    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4641d0dc_3dca_4066_8222_3d9b59dd2658.slice/crio-887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a WatchSource:0}: Error finding container 887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a: Status 404 returned error can't find the container with id 887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a
Nov 02 15:13:04.502433 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:04.502397    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:04.502542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:04.502475    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:04.502542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:04.502505    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:05.125362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:05.125323    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" event={"ID":"4641d0dc-3dca-4066-8222-3d9b59dd2658","Type":"ContainerStarted","Data":"b05236150165be48ae5cfb5b99479f708facd6e634e911aa5dd314b3df0f65ca"}
Nov 02 15:13:05.125839 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:05.125384    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" event={"ID":"4641d0dc-3dca-4066-8222-3d9b59dd2658","Type":"ContainerStarted","Data":"887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a"}
Nov 02 15:13:05.347045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:05.346984    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podStartSLOduration=2.346968297 podStartE2EDuration="2.346968297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:13:05.295352796 +0000 UTC m=+991.733254938" watchObservedRunningTime="2026-11-02 15:13:05.346968297 +0000 UTC m=+991.784870513"
Nov 02 15:13:06.056791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:06.056753    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:06.058603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:06.058574    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:06.058774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:06.058625    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:07.056628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:07.056590    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:07.057184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:07.056645    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:08.056791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:08.056748    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:08.057245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:08.056830    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:09.056624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:09.056580    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:09.056822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:09.056655    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:10.056419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:10.056375    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:10.056654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:10.056433    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:11.061216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:11.060590    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 02 15:13:11.061216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:11.060650    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 02 15:13:12.356966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:12.356913    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:12.357663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:12.357635    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:12.358787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:12.358761    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:13:33.279108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.279062    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:13:33.283499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.283474    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381590    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381656    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381688    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381721    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381754    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.381860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381841    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz5kg\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.382148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381873    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.382148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.381986    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.382148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.382081    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.390021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.389989    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:13:33.483344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483298    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483358    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483388    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483421    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483559    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483674    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz5kg\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483802 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483713    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483832    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.483968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.483934    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.484114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.484083    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.484277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.484168    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.484277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.484260    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.484418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.484370    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.484418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.484387    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.485830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.485811    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.486168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.486148    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.545096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.545012    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz5kg\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.545096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.545013    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:33.567220 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:13:33.567161    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:13:33.598807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:33.598766    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:34.036884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.036859    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:13:34.038888 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:13:34.038856    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ddc1efd_e33d_44cc_862e_b2b8abee1c20.slice/crio-93a1b2d2223c32aca70b60602a614894a3b67db44203ea6592490e876e9f15a9 WatchSource:0}: Error finding container 93a1b2d2223c32aca70b60602a614894a3b67db44203ea6592490e876e9f15a9: Status 404 returned error can't find the container with id 93a1b2d2223c32aca70b60602a614894a3b67db44203ea6592490e876e9f15a9
Nov 02 15:13:34.041682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.041648    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:34.041771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.041716    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:34.041771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.041742    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:13:34.309326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.309206    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" event={"ID":"8ddc1efd-e33d-44cc-862e-b2b8abee1c20","Type":"ContainerStarted","Data":"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"}
Nov 02 15:13:34.309326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.309317    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" event={"ID":"8ddc1efd-e33d-44cc-862e-b2b8abee1c20","Type":"ContainerStarted","Data":"93a1b2d2223c32aca70b60602a614894a3b67db44203ea6592490e876e9f15a9"}
Nov 02 15:13:34.599155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:34.599055    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:35.601138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:35.601103    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:35.601632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:35.601158    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:36.599771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:36.599733    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:36.599949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:36.599802    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:37.599869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:37.599826    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:37.600397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:37.599897    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:38.599829 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:38.599789    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:38.600024 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:38.599855    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:39.599815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:39.599770    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:39.601850 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:39.599840    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:40.600141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:40.600093    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 02 15:13:40.600375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:40.600170    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 02 15:13:41.604987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:41.604952    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:41.605492 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:41.605297    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:41.606035 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:41.606017    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:13:41.780505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:13:41.780446    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podStartSLOduration=8.780429855 podStartE2EDuration="8.780429855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:13:34.447586068 +0000 UTC m=+1020.885488268" watchObservedRunningTime="2026-11-02 15:13:41.780429855 +0000 UTC m=+1028.218331995"
Nov 02 15:14:00.329284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.329250    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:14:00.333280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.333262    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.391505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.391466    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.391680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.391520    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnjj\" (UniqueName: \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.391680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.391626    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.391680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.391665    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.391841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.391689    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.392245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.392221    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.435207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.435168    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-de35cd\"/\"openshift-service-ca.crt\""
Nov 02 15:14:00.435435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.435169    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-de35cd\"/\"kube-root-ca.crt\""
Nov 02 15:14:00.435435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.435232    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-de35cd\"/\"default-dockercfg-w8wnr\""
Nov 02 15:14:00.435435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.435286    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-de35cd\"/\"distinct-w2-de35cd-kserve-self-signed-certs\""
Nov 02 15:14:00.438211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.438176    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:14:00.495091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.495041    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.495444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.495419    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.495601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.495582    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.495876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.495829    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.507211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.496165    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.508395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.508369    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnjj\" (UniqueName: \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.512137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.507967    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.513787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.513761    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.515496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.515470    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.538238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.537541    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.546217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.543240    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.569845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.569809    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnjj\" (UniqueName: \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\") pod \"distinct-w2-de35cd-kserve-c459c89b7-jwbtj\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:00.646861 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:00.646753    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:01.066262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:01.066236    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:14:01.068099 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:14:01.068072    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a915c87_81cc_4937_b9fc_c7d78ff67832.slice/crio-13c01a51daf7776cf2831429be6590752caf87073c28813bb1d93adaa47e2d26 WatchSource:0}: Error finding container 13c01a51daf7776cf2831429be6590752caf87073c28813bb1d93adaa47e2d26: Status 404 returned error can't find the container with id 13c01a51daf7776cf2831429be6590752caf87073c28813bb1d93adaa47e2d26
Nov 02 15:14:01.542126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:01.542085    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" event={"ID":"9a915c87-81cc-4937-b9fc-c7d78ff67832","Type":"ContainerStarted","Data":"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"}
Nov 02 15:14:01.542561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:01.542147    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" event={"ID":"9a915c87-81cc-4937-b9fc-c7d78ff67832","Type":"ContainerStarted","Data":"13c01a51daf7776cf2831429be6590752caf87073c28813bb1d93adaa47e2d26"}
Nov 02 15:14:01.647827 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:01.647787    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:01.660550 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:01.660495    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podStartSLOduration=1.660478355 podStartE2EDuration="1.660478355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:14:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:14:01.658611939 +0000 UTC m=+1048.096514073" watchObservedRunningTime="2026-11-02 15:14:01.660478355 +0000 UTC m=+1048.098380495"
Nov 02 15:14:03.566065 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:14:03.566027    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:14:10.646917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:10.646881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:10.659660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:10.659630    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:11.659216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:11.659176    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:14:20.375692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.375645    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:14:20.380253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.380227    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.477155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.477108    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:14:20.528573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528540    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.528573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528575    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.528814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528604    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.528852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528814    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.528899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528863    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tz6v\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.528985 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.528965    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.529050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.529001    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.529050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.529037    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.529139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.529126    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630275    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630316    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630336    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630360    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630434    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630450    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630482    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630669    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.630853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.630740    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tz6v\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.631041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.631012    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.631170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.631091    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.631170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.631031    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.631285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.631183    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.631412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.631392    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.633226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.633206    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.633553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.633531    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.697137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.697105    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tz6v\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.697137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.697107    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:20.994848 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:20.994816    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:21.428952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.428911    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:14:21.430219 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:14:21.430173    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ea1a00_4254_44b4_9e3d_ca1a322e89de.slice/crio-8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5 WatchSource:0}: Error finding container 8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5: Status 404 returned error can't find the container with id 8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5
Nov 02 15:14:21.432751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.432711    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:14:21.432841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.432795    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:14:21.432841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.432826    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:14:21.712889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.712848    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" event={"ID":"62ea1a00-4254-44b4-9e3d-ca1a322e89de","Type":"ContainerStarted","Data":"d069bf55a2604e9f00f408f56096080c4a56f59a5adde685dda533bc39865147"}
Nov 02 15:14:21.713098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.712920    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" event={"ID":"62ea1a00-4254-44b4-9e3d-ca1a322e89de","Type":"ContainerStarted","Data":"8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5"}
Nov 02 15:14:21.839960 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.839873    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podStartSLOduration=1.839853762 podStartE2EDuration="1.839853762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:14:21.83588331 +0000 UTC m=+1068.273785489" watchObservedRunningTime="2026-11-02 15:14:21.839853762 +0000 UTC m=+1068.277755902"
Nov 02 15:14:21.997473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:21.997367    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:23.011214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:23.000555    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:23.011214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:23.000636    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:23.996679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:23.996635    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:23.997687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:23.997653    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:24.995889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:24.995843    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:24.996339 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:24.995915    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:25.996215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:25.995930    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:25.996215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:25.996015    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:26.995837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:26.995791    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:26.996064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:26.995865    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:27.995574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:27.995532    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Nov 02 15:14:27.996003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:27.995589    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Nov 02 15:14:29.001459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:29.001421    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:29.001955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:29.001860    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:29.002688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:29.002668    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:14:33.565205 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:14:33.565153    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:14:34.125707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.125671    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:14:34.129797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.129779    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216603    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwpzj\" (UniqueName: \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216655    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216786    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216811    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216839    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.216947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.216907    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.231279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.231234    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-6dc69b\"/\"openshift-service-ca.crt\""
Nov 02 15:14:34.231451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.231347    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6dc69b\"/\"default-dockercfg-j445d\""
Nov 02 15:14:34.231451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.231423    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-6dc69b\"/\"kube-root-ca.crt\""
Nov 02 15:14:34.231581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.231530    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-6dc69b\"/\"distinct-w4-6dc69b-kserve-self-signed-certs\""
Nov 02 15:14:34.235459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.235431    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:14:34.328245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328202    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328494    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328556    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328595    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328739    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328768    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwpzj\" (UniqueName: \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.328878 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328818    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.329016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.328994    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.329465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.329309    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.335093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.335058    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.341143 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.341111    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.390272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.390170    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwpzj\" (UniqueName: \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\") pod \"distinct-w4-6dc69b-kserve-6676c8bfdf-hf626\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") " pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.443000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.442962    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:34.882326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:34.882300    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:14:34.883324 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:14:34.883293    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod629aba4f_7644_4aec_88d3_f4559b6d3179.slice/crio-6b59767bb583842af61ebe136a99dc6982bfc7838e53a67a8da62e3a6b6a45fe WatchSource:0}: Error finding container 6b59767bb583842af61ebe136a99dc6982bfc7838e53a67a8da62e3a6b6a45fe: Status 404 returned error can't find the container with id 6b59767bb583842af61ebe136a99dc6982bfc7838e53a67a8da62e3a6b6a45fe
Nov 02 15:14:35.812513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:35.812473    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" event={"ID":"629aba4f-7644-4aec-88d3-f4559b6d3179","Type":"ContainerStarted","Data":"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"}
Nov 02 15:14:35.812513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:35.812517    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" event={"ID":"629aba4f-7644-4aec-88d3-f4559b6d3179","Type":"ContainerStarted","Data":"6b59767bb583842af61ebe136a99dc6982bfc7838e53a67a8da62e3a6b6a45fe"}
Nov 02 15:14:35.930168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:35.930101    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podStartSLOduration=1.930084473 podStartE2EDuration="1.930084473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:14:35.92914638 +0000 UTC m=+1082.367048521" watchObservedRunningTime="2026-11-02 15:14:35.930084473 +0000 UTC m=+1082.367986616"
Nov 02 15:14:36.443996 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:36.443965    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:44.443970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:44.443931    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:44.456991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:44.456964    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:14:44.877381 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:14:44.877292    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:15:03.566431 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:15:03.566392    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:15:10.153403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.153368    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"]
Nov 02 15:15:10.157575 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.157556    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.234857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.234814    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.235103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.235081    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.235258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.235236    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnkgt\" (UniqueName: \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.259333 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.259288    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 15:15:10.259333 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.259309    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pzqc\""
Nov 02 15:15:10.262326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.262304    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"]
Nov 02 15:15:10.336763 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.336724    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnkgt\" (UniqueName: \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.336964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.336804    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.336964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.336924    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.337676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.337650    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.339401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.339383    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.397638 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.397602    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnkgt\" (UniqueName: \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\") pod \"maas-api-revoke-sub-a8d31fdd-5hk9g\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.472184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.472152    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:10.850532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.850508    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"]
Nov 02 15:15:10.852168 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:15:10.852137    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode070147d_3f97_41b0_a487_cdab361c9274.slice/crio-e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845 WatchSource:0}: Error finding container e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845: Status 404 returned error can't find the container with id e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845
Nov 02 15:15:10.854458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:10.854443    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:15:11.028433 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:11.028399    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g" event={"ID":"e070147d-3f97-41b0-a487-cdab361c9274","Type":"ContainerStarted","Data":"e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845"}
Nov 02 15:15:15.100208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:15.099325    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g" event={"ID":"e070147d-3f97-41b0-a487-cdab361c9274","Type":"ContainerStarted","Data":"466a1659b8a0df608156befe1be51fa35e16d38a61b7c6e18b0eb6a7e2ae54a9"}
Nov 02 15:15:15.218940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:15.218885    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g" podStartSLOduration=1.747072326 podStartE2EDuration="5.218870843s" totalImagesPullingTime="3.471798517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:15:15.218405642 +0000 UTC m=+1121.656307784" watchObservedRunningTime="2026-11-02 15:15:15.218870843 +0000 UTC m=+1121.656772983"
Nov 02 15:15:16.107217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:16.107156    4470 generic.go:356] "Generic (PLEG): container finished" podID="e070147d-3f97-41b0-a487-cdab361c9274" containerID="466a1659b8a0df608156befe1be51fa35e16d38a61b7c6e18b0eb6a7e2ae54a9" exitCode=0
Nov 02 15:15:16.107743 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:16.107222    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g" event={"ID":"e070147d-3f97-41b0-a487-cdab361c9274","Type":"ContainerDied","Data":"466a1659b8a0df608156befe1be51fa35e16d38a61b7c6e18b0eb6a7e2ae54a9"}
Nov 02 15:15:17.275498 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.275470    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:17.373224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.373112    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\") pod \"e070147d-3f97-41b0-a487-cdab361c9274\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") "
Nov 02 15:15:17.373382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.373356    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\") pod \"e070147d-3f97-41b0-a487-cdab361c9274\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") "
Nov 02 15:15:17.373450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.373438    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\" (UniqueName: \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\") pod \"e070147d-3f97-41b0-a487-cdab361c9274\" (UID: \"e070147d-3f97-41b0-a487-cdab361c9274\") "
Nov 02 15:15:17.373532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.373507    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca" pod "e070147d-3f97-41b0-a487-cdab361c9274" (UID: "e070147d-3f97-41b0-a487-cdab361c9274"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:15:17.373777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.373761    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-openshift-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:15:17.375590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.375570    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt" pod "e070147d-3f97-41b0-a487-cdab361c9274" (UID: "e070147d-3f97-41b0-a487-cdab361c9274"). InnerVolumeSpecName "kube-api-access-tnkgt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:15:17.375668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.375646    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca" pod "e070147d-3f97-41b0-a487-cdab361c9274" (UID: "e070147d-3f97-41b0-a487-cdab361c9274"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:15:17.475564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.475527    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e070147d-3f97-41b0-a487-cdab361c9274-maas-api-xks-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:15:17.475564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:17.475562    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnkgt\" (UniqueName: \"kubernetes.io/projected/e070147d-3f97-41b0-a487-cdab361c9274-kube-api-access-tnkgt\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:15:18.121127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:18.121046    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g" event={"ID":"e070147d-3f97-41b0-a487-cdab361c9274","Type":"ContainerDied","Data":"e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845"}
Nov 02 15:15:18.121127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:18.121079    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"
Nov 02 15:15:18.121127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:18.121095    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e34461c043cbd6bafe707c3bc2f7fcb7d56b0cd030292c51931cc71c164db845"
Nov 02 15:15:33.565869 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:15:33.565829    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:15:44.252652 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.252616    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:15:44.253182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.253169    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e070147d-3f97-41b0-a487-cdab361c9274" containerName="revoke-keys"
Nov 02 15:15:44.253182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.253182    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e070147d-3f97-41b0-a487-cdab361c9274" containerName="revoke-keys"
Nov 02 15:15:44.253315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.253302    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e070147d-3f97-41b0-a487-cdab361c9274" containerName="revoke-keys"
Nov 02 15:15:44.256722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.256707    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.326510 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.326473    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:15:44.368124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368089    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fvcr\" (UniqueName: \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.368320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368155    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.368320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368271    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.368320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368298    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.368452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368321    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.368534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.368513    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.372546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.372520    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c18e79\"/\"openshift-service-ca.crt\""
Nov 02 15:15:44.372546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.372538    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c18e79\"/\"default-dockercfg-nmwlk\""
Nov 02 15:15:44.372704 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.372620    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-c18e79\"/\"distinct-2-w6-c18e79-kserve-self-signed-certs\""
Nov 02 15:15:44.373509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.373490    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-c18e79\"/\"kube-root-ca.crt\""
Nov 02 15:15:44.470903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.470866    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fvcr\" (UniqueName: \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.471607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.471575    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.472443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.472411    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.472625 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.472606    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.472766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.472751    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.472993 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.472979    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.474756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.473570    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.474756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.474285    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.474756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.474535    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.475925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.475902    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.481776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.481747    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.570862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.570784    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fvcr\" (UniqueName: \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\") pod \"distinct-2-w6-c18e79-kserve-695df687f5-kchkv\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") " pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:44.869764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:44.869682    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:45.250764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:45.250725    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:15:45.252132 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:15:45.252098    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd40061a_9e41_4636_9e48_3deac85661d1.slice/crio-980d3ca27c0074b917abe34b3471adaf5d8fe73a9e6666e545146b8af5321e2b WatchSource:0}: Error finding container 980d3ca27c0074b917abe34b3471adaf5d8fe73a9e6666e545146b8af5321e2b: Status 404 returned error can't find the container with id 980d3ca27c0074b917abe34b3471adaf5d8fe73a9e6666e545146b8af5321e2b
Nov 02 15:15:45.316915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:45.316884    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" event={"ID":"dd40061a-9e41-4636-9e48-3deac85661d1","Type":"ContainerStarted","Data":"980d3ca27c0074b917abe34b3471adaf5d8fe73a9e6666e545146b8af5321e2b"}
Nov 02 15:15:46.324221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:46.324167    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" event={"ID":"dd40061a-9e41-4636-9e48-3deac85661d1","Type":"ContainerStarted","Data":"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"}
Nov 02 15:15:46.441499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:46.441352    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podStartSLOduration=2.44133701 podStartE2EDuration="2.44133701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:15:46.44087549 +0000 UTC m=+1152.878777630" watchObservedRunningTime="2026-11-02 15:15:46.44133701 +0000 UTC m=+1152.879239151"
Nov 02 15:15:46.870764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:46.870721    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:54.870237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:54.870184    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:54.883622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:54.883591    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:15:55.419668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:15:55.419638    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:16:03.568048 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:16:03.568002    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:16:19.967409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:19.967372    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:16:19.971673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:19.971651    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057075 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057039    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057119    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057262    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057315    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057398    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057440    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057510    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057554    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.057626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.057611    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs578\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159040    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159109    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159217    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159252    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159491 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159323    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159491 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159348    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159496    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159563    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159568    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159597    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.159736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.159617    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs578\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.160052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.160024    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.160257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.160169    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.160436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.160416    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.161663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.161638    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.161794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.161775    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.178031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.177997    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:16:20.221283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.221172    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.221532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.221259    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs578\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.286611 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.286576    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:20.325337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.325305    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:16:20.329735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.329707    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.434905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.434871    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:16:20.464422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464389    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464460    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464570    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464595    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464783    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464842    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.464973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.464948    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.465058 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.465018    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.465111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.465073    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dshqs\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.566841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.566766    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dshqs\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.566841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.566834    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.566908    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.566967    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.566996    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567155    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567244    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567271    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567281    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567311    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567322    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567559    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567620    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.567955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.567928    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.569829 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.569807    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.569978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.569958    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.629111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.629082    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.629513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.629491    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dshqs\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.650941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.650909    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:20.729498 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.729463    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:16:20.735088 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:16:20.735058    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ed6bbc_337d_431b_bc04_1736db28c879.slice/crio-aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1 WatchSource:0}: Error finding container aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1: Status 404 returned error can't find the container with id aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1
Nov 02 15:16:20.737629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.737587    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:20.737761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.737678    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:20.737761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:20.737723    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:21.034860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.034837    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:16:21.037043 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:16:21.037014    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10572e2d_8698_4f35_8c23_8437bbd65edb.slice/crio-2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780 WatchSource:0}: Error finding container 2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780: Status 404 returned error can't find the container with id 2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780
Nov 02 15:16:21.039908 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.039866    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:21.040018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.039962    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:21.040018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.040012    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:21.606396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.606359    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" event={"ID":"e3ed6bbc-337d-431b-bc04-1736db28c879","Type":"ContainerStarted","Data":"16ab0570d68f48dfc2508d83e562ee0d55f26fca8cfbdd0a2e621a568447568d"}
Nov 02 15:16:21.606584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.606419    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" event={"ID":"e3ed6bbc-337d-431b-bc04-1736db28c879","Type":"ContainerStarted","Data":"aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1"}
Nov 02 15:16:21.608091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.608067    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" event={"ID":"10572e2d-8698-4f35-8c23-8437bbd65edb","Type":"ContainerStarted","Data":"c35baba076ca88fcf4a32a2eb96269366d9cbc3934e0cf22624f4568679bf078"}
Nov 02 15:16:21.608227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.608105    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" event={"ID":"10572e2d-8698-4f35-8c23-8437bbd65edb","Type":"ContainerStarted","Data":"2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780"}
Nov 02 15:16:21.651811 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.651778    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:21.734258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.734173    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podStartSLOduration=2.734151724 podStartE2EDuration="2.734151724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:16:21.731438532 +0000 UTC m=+1188.169340673" watchObservedRunningTime="2026-11-02 15:16:21.734151724 +0000 UTC m=+1188.172053868"
Nov 02 15:16:21.806102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.806064    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:16:21.822930 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.822886    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:21.863056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.862947    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:16:21.863368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:21.863318    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podStartSLOduration=1.863300426 podStartE2EDuration="1.863300426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:16:21.86014363 +0000 UTC m=+1188.298045773" watchObservedRunningTime="2026-11-02 15:16:21.863300426 +0000 UTC m=+1188.301202568"
Nov 02 15:16:22.017212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017005    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017075    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs5w2\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017106    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017152    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017290    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017328    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017433    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017499    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.017595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.017528    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.126182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.124767    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.126182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.125530    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.126182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.125961    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.126800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.126212    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127028    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127042    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127098    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127170    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127244    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs5w2\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127275    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127323    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.127481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.127371    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.128136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.128105    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.128407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.128345    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.129042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.129025    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.130403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.130378    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.218221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.218117    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs5w2\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.220862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.220805    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.295213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.294474    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:22.295213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.294696    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:22.295213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.294766    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:22.440315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.440286    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:22.653541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.653507    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:22.653733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.653559    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:22.946579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.946553    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:16:22.947586 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:16:22.947554    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6d5321_f8fc_4d72_a2a9_28d2eef87979.slice/crio-46771254711f2715dfba6679649933941a8e668461fae565b4e06c1fda2c44f5 WatchSource:0}: Error finding container 46771254711f2715dfba6679649933941a8e668461fae565b4e06c1fda2c44f5: Status 404 returned error can't find the container with id 46771254711f2715dfba6679649933941a8e668461fae565b4e06c1fda2c44f5
Nov 02 15:16:22.950248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.950210    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:22.950320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.950290    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:22.950354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:22.950323    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:23.301807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.301710    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:23.302383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.302348    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:23.645119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.645029    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" event={"ID":"4d6d5321-f8fc-4d72-a2a9-28d2eef87979","Type":"ContainerStarted","Data":"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"}
Nov 02 15:16:23.645119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.645086    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" event={"ID":"4d6d5321-f8fc-4d72-a2a9-28d2eef87979","Type":"ContainerStarted","Data":"46771254711f2715dfba6679649933941a8e668461fae565b4e06c1fda2c44f5"}
Nov 02 15:16:23.651717 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.651690    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:23.651874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:23.651736    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:24.308884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.308845    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:24.309521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.309487    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:24.442890 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.442845    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:24.447347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.447309    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:24.447595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.447569    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:24.652150 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.652068    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:24.652386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.652363    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:24.778516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.778445    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podStartSLOduration=3.778422138 podStartE2EDuration="3.778422138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:16:23.789591092 +0000 UTC m=+1190.227493246" watchObservedRunningTime="2026-11-02 15:16:24.778422138 +0000 UTC m=+1191.216324282"
Nov 02 15:16:24.815214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.804231    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:16:24.821571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.821542    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.865009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.864978    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908026    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9xwm\" (UniqueName: \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908218    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908266    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908345    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908416    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908457    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:24.938302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:24.908919    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-de35cd\"/\"distinct-2-w2-de35cd-kserve-self-signed-certs\""
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025002    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025071    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025202    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9xwm\" (UniqueName: \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025345    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025385    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025463    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.027212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.025751    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.099213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.080471    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.099431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.099406    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.227997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.227868    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.314224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.300602    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:25.314224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.300664    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:25.446332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.444054    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:25.446332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.444112    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:25.446332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.444561    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9xwm\" (UniqueName: \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.468074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.451262    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\") pod \"distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") " pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.554880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.540484    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:25.653125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.652671    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:25.653125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:25.652733    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:26.446440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.431479    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:26.446440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.431538    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:26.446440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.443663    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:26.446440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.443721    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:26.534227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.518336    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:16:26.656752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.656652    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:26.657043 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.657010    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:26.746572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:26.746538    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" event={"ID":"679279a9-76bb-49aa-a850-04469ea87fcf","Type":"ContainerStarted","Data":"df517c7ea474834bd35cfec13b2fe7d03703818206e3e68d861cf081bf11333c"}
Nov 02 15:16:27.289132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.288636    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:27.289132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.288720    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:27.447125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.446643    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:27.447125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.446709    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:27.651745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.651653    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:27.652003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.651975    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:27.779214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.770202    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" event={"ID":"679279a9-76bb-49aa-a850-04469ea87fcf","Type":"ContainerStarted","Data":"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"}
Nov 02 15:16:27.915402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:27.908430    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podStartSLOduration=3.908408639 podStartE2EDuration="3.908408639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:16:27.907420025 +0000 UTC m=+1194.345322167" watchObservedRunningTime="2026-11-02 15:16:27.908408639 +0000 UTC m=+1194.346310781"
Nov 02 15:16:28.290532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.290489    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:28.290909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.290870    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:28.458841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.445638    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:28.458841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.445708    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:28.552276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.551288    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:28.665217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.657624    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 02 15:16:28.665217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:28.657688    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 02 15:16:29.290216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:29.289648    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 02 15:16:29.290216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:29.289714    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 02 15:16:29.441296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:29.441258    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:29.441527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:29.441327    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:30.293160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.293128    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:30.293713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.293614    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:30.294483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.294460    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:16:30.441174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.441127    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Nov 02 15:16:30.441398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.441184    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Nov 02 15:16:30.692392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.692355    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:16:30.692693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:30.692649    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:16:31.327420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.327379    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:31.328308 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.328279    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:31.329330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.329302    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:16:31.509108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.509074    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:31.509578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.509554    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:31.510483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:31.510464    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:16:33.565776 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:16:33.565733    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:16:33.836350 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:33.836269    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:16:33.847578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:33.847554    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:16:35.544862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:35.544816    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:35.558110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:35.558078    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:35.937172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:35.937146    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:16:51.556395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.556320    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:16:51.563091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.563068    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.604684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.604648    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.604863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.604789    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.604863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.604821    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.604863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.604859    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.604998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.604982    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96rlg\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.605037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.605026    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.605070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.605050    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.605119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.605099    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.605225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.605213    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706358 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706327    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706416    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706442    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706485    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706541    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96rlg\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706590    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706620    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706645    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706700    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.706809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.706776    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.707226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.707168    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.707686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.707656    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.707955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.707931    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.708029 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.707960    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.709178 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.709147    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.710252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.710230    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.770520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.770482    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96rlg\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.770704 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.770524    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:51.771229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.771205    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:16:51.876711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:51.876625    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:52.316062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:52.316036    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:16:52.318333 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:16:52.318304    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa1e379_8d3b_4359_8fba_257fd79483c4.slice/crio-ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b WatchSource:0}: Error finding container ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b: Status 404 returned error can't find the container with id ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b
Nov 02 15:16:52.320700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:52.320666    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:52.320767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:52.320733    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:52.320767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:52.320762    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:16:53.034180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.034143    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" event={"ID":"4aa1e379-8d3b-4359-8fba-257fd79483c4","Type":"ContainerStarted","Data":"a97ab39269b2a1ea61a8bee2a9a6ffa01f30cc7d8d5c86f058e4cc22065b8873"}
Nov 02 15:16:53.034615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.034246    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" event={"ID":"4aa1e379-8d3b-4359-8fba-257fd79483c4","Type":"ContainerStarted","Data":"ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b"}
Nov 02 15:16:53.207419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.207351    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podStartSLOduration=2.20732768 podStartE2EDuration="2.20732768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:16:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:16:53.204575501 +0000 UTC m=+1219.642477644" watchObservedRunningTime="2026-11-02 15:16:53.20732768 +0000 UTC m=+1219.645229822"
Nov 02 15:16:53.889214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.881582    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:16:53.895887 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.895844    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:53.896072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:53.895912    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:16:54.893214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:54.892326    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:54.893214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:54.892403    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:16:55.884224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:55.879695    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:55.884224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:55.879977    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:16:56.894684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:56.894608    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:56.897360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:56.897314    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:16:57.878219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:57.877557    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:57.878219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:57.877623    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:16:58.877758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:58.877717    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Nov 02 15:16:58.878219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:16:58.877780    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Nov 02 15:17:00.130823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:00.130784    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:17:00.131592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:00.131567    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:17:00.132859 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:00.132835    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:17:02.559370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:02.559326    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:17:02.560016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:02.559971    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:02.704215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:02.681549    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:03.566607 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:17:03.566572    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:17:20.129272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.129237    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:17:20.133583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.133559    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.179955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.179923    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:17:20.184218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.184161    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.229047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.229015    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:17:20.243529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.243486    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:17:20.294708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294671    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.294920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294717    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.294920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294823    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.294920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294859    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.294920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294895    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294923    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlkb6\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.294974    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295029    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295088    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295121    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295156    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhkh5\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295228    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295292    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295347    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295414    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.295699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295466    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295487    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.295699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.295530    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400773    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400846    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400880    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400906    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400962    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.400995    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401061    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401094    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401140    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401179    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlkb6\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401239    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401276    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401350    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401374    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401399    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhkh5\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401429    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401514    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.401562    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.402322    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.402578    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.402961    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.403251    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.403251    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.403847    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.403879    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.404137    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.404345    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.404769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.404528    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.418053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.410800    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.418053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.412167    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.418053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.412628    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.435373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.432013    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.470487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.470455    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.470643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.470571    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlkb6\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.474743 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.474704    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.475363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.475341    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhkh5\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\") pod \"e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") " pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.499279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.499234    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:20.760243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.760180    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:20.804928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.804397    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:17:20.810254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.810227    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.846555 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.846521    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:17:20.851763 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.851739    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:20.881287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.881253    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:17:20.908702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.908665    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.908923 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.908715    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.908940    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdr6m\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.908974    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.909105    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.909170    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909338 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.909316    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.909384    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.909503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.909484    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:20.918450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.918416    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:17:20.928173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.926007    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:17:20.928344 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:17:20.928292    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349f2413_a6d8_483a_be3b_4b496709914c.slice/crio-244fee10ab606c3e746d2cfdabda23c005293422a2a71447cae035e3dd35c52d WatchSource:0}: Error finding container 244fee10ab606c3e746d2cfdabda23c005293422a2a71447cae035e3dd35c52d: Status 404 returned error can't find the container with id 244fee10ab606c3e746d2cfdabda23c005293422a2a71447cae035e3dd35c52d
Nov 02 15:17:20.931871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.931829    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:20.931978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.931916    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:20.931978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:20.931963    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.010800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.010715    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.010800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.010776    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.010859    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.010920    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.010970    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c64vv\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011023    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011160    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011233    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011280    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011317    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011334    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011357    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011600 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011561    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011613    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011641    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.011758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011693    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011723    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011789    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdr6m\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011821    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.011864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.011852    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.012031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.012008    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.012499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.012473    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.012619 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.012505    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.013248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.013228    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.014522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.014499    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.076836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.076800    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.078234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.078174    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdr6m\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.113269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113227    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.113446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113288    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.113446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113342    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.113597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113455    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.113597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113544    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113604    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113646    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113700    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c64vv\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113716    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113724    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.113834    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.114049    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.114147    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.114833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.114456    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.116125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.116104    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.116276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.116252    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.129533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.129502    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:21.152524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.152497    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:17:21.153084 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:17:21.153063    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7022fdf6_b3a9_41c2_b03d_7dc044256591.slice/crio-29f635222db3a006d7eceda2a9743c2e96627b615ac75bf9e3c81400a1d6c01f WatchSource:0}: Error finding container 29f635222db3a006d7eceda2a9743c2e96627b615ac75bf9e3c81400a1d6c01f: Status 404 returned error can't find the container with id 29f635222db3a006d7eceda2a9743c2e96627b615ac75bf9e3c81400a1d6c01f
Nov 02 15:17:21.156399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.156355    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.156503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.156451    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.156557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.156498    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.175139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.175109    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.177584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.177557    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c64vv\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\") pod \"e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") " pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.340879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.339982    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" event={"ID":"349f2413-a6d8-483a-be3b-4b496709914c","Type":"ContainerStarted","Data":"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"}
Nov 02 15:17:21.340879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.340050    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" event={"ID":"349f2413-a6d8-483a-be3b-4b496709914c","Type":"ContainerStarted","Data":"244fee10ab606c3e746d2cfdabda23c005293422a2a71447cae035e3dd35c52d"}
Nov 02 15:17:21.342167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.342135    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" event={"ID":"7022fdf6-b3a9-41c2-b03d-7dc044256591","Type":"ContainerStarted","Data":"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"}
Nov 02 15:17:21.342312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.342247    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" event={"ID":"7022fdf6-b3a9-41c2-b03d-7dc044256591","Type":"ContainerStarted","Data":"29f635222db3a006d7eceda2a9743c2e96627b615ac75bf9e3c81400a1d6c01f"}
Nov 02 15:17:21.465047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.464948    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podStartSLOduration=1.464931741 podStartE2EDuration="1.464931741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:21.460458515 +0000 UTC m=+1247.898360762" watchObservedRunningTime="2026-11-02 15:17:21.464931741 +0000 UTC m=+1247.902833882"
Nov 02 15:17:21.466984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.466951    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:21.499742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.499626    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:21.541287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.541174    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:17:21.547004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.546484    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.547004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.546590    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.547004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.546639    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.623671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.623609    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podStartSLOduration=1.623584718 podStartE2EDuration="1.623584718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:21.599765057 +0000 UTC m=+1248.037667200" watchObservedRunningTime="2026-11-02 15:17:21.623584718 +0000 UTC m=+1248.061486862"
Nov 02 15:17:21.760523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.760482    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:21.915552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.915451    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:17:21.941424 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.941369    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.941834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.941792    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:21.942150 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:21.942130    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:22.367978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.366956    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" event={"ID":"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784","Type":"ContainerStarted","Data":"b58fe1f21700dbe6c5cea687cda1b749e4cd49d8d7f8afcdc22e98947050b219"}
Nov 02 15:17:22.367978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.367041    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" event={"ID":"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784","Type":"ContainerStarted","Data":"929891014ff173247ed78682e3a24fb7c78d5105c3bcf0069f7d730409f896ca"}
Nov 02 15:17:22.392226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.382923    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" event={"ID":"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7","Type":"ContainerStarted","Data":"5232b0e381fca051d9eb7a9f4219d24fd50633c875f953fc0c4127ddccf0fbab"}
Nov 02 15:17:22.392639 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.392589    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" event={"ID":"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7","Type":"ContainerStarted","Data":"250dd917e77e59e6e3a482695456790e7a5233cb38925a262f510903deaf7ef9"}
Nov 02 15:17:22.521217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.520419    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:22.543232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.522817    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:22.543232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.522873    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:22.630395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.630274    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:17:22.635546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.635508    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:17:22.635753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.635647    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.638859 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.638790    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podStartSLOduration=2.638768861 podStartE2EDuration="2.638768861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:22.5196708 +0000 UTC m=+1248.957572942" watchObservedRunningTime="2026-11-02 15:17:22.638768861 +0000 UTC m=+1249.076671003"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759723    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759784    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759815    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759848    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759879    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759956    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.759984    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.760016    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl47j\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.760213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.760049    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.769427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.769390    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:22.775324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.775275    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:22.821394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.794354    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podStartSLOduration=2.794330527 podStartE2EDuration="2.794330527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:22.780087119 +0000 UTC m=+1249.217989261" watchObservedRunningTime="2026-11-02 15:17:22.794330527 +0000 UTC m=+1249.232232669"
Nov 02 15:17:22.940976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.940929    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941248    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941411    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941544    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941663    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941838    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.941981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.941965    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.942102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.942084    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl47j\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.942247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.942232    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.996214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.942428    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.996536 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.962620    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:22.996667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.963516    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.000212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.963721    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.010443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:22.973908    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.010755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.009931    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.050284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.028583    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.050767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.050723    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.104216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.096427    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl47j\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.136631 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.136592    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:23.139232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.139177    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:23.139484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.139455    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:23.220868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.220502    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:23.484641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.484550    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:23.504729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.504670    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:23.520856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.519358    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:23.520856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.519432    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:23.774278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.772762    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:23.774278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:23.772822    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:24.325225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.248474    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:24.325225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.248538    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:24.482242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.480902    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:24.503529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.495315    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:24.503529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.495382    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:24.519220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.518092    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:24.519220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.518155    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:24.578328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.578147    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:24.578693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.578665    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:24.579088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.579066    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:24.716231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.701276    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:24.716231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.701863    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:17:24.775369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.775322    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:24.775637 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.775606    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:24.794218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.777316    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" event={"ID":"b716ef3d-edfe-4b55-b337-18e87e77d4ae","Type":"ContainerStarted","Data":"3a0bcb72fee7936e1e4321ef7117e44a42afe0ead2ff1f1fdb1fea731f0782f0"}
Nov 02 15:17:24.918229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.916704    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:24.947240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.922855    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:24.971214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:24.950868    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:17:25.116213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.115755    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:25.229214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.208430    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:25.229214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.208493    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:25.518245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.492176    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:25.518245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.492606    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:17:25.534242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.519481    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:25.534242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.519625    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:25.534242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.519661    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:25.534242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.521219    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:25.534242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.521259    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:25.951273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.925391    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:25.951273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.935579    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:25.951520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.951473    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:25.991660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:25.991614    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:17:26.002500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.002466    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:26.010301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.010259    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:26.101221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.075859    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": context deadline exceeded"
Nov 02 15:17:26.140280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.131485    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:17:26.255311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.176040    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:26.255311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.247371    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:26.392677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.392644    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:17:26.393060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.393029    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:17:26.393403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.393380    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:17:26.401230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.395761    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:17:26.401230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.395918    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:17:26.521269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.514690    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:26.521269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.514764    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:26.521269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.514839    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:26.555604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.555568    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:26.559509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.559484    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:26.574535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.559402    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Nov 02 15:17:26.577544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.577507    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" event={"ID":"b716ef3d-edfe-4b55-b337-18e87e77d4ae","Type":"ContainerStarted","Data":"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"}
Nov 02 15:17:26.803665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.803562    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:26.803945 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.803898    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:26.976769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:26.972792    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.052217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.017591    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.056346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.018304    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.179215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.178296    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:27.179215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.178361    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:27.225317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.225145    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:27.235010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.234746    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:27.235010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.234807    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:27.248215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.247263    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.425955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.425853    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.480215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.474804    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:27.480215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.474862    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:27.537217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.523180    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:27.537217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.530117    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:27.537217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.530208    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:27.755220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.753672    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podStartSLOduration=5.753647947 podStartE2EDuration="5.753647947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:27.309775686 +0000 UTC m=+1253.747677829" watchObservedRunningTime="2026-11-02 15:17:27.753647947 +0000 UTC m=+1254.191550089"
Nov 02 15:17:27.812374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.812235    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:27.812374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:27.812308    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:28.050274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.041880    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.046995    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.111463    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.304592    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.304633    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.306264    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.358351    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.351675    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:28.365271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.358413    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:28.469376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.469319    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:28.541217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.537496    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:28.541217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.537559    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:28.541217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.537642    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:28.541217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.537665    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:28.541217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.538344    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:28.896305 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:28.875589    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.060409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.060364    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:29.365026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.363436    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.365026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.363506    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:29.386006 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.385950    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:29.386726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.386688    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:29.387254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.387229    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:29.476216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.475046    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:29.476216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.390100    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.476216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.475529    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:29.477563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.477527    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.477714 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.477580    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:29.499977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.499919    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:17:29.577214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.533769    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.625215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.625106    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Nov 02 15:17:29.625465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.546042    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:29.831216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.827448    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Nov 02 15:17:29.831216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:29.827532    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Nov 02 15:17:30.181212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.168996    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:30.181212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.169055    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:30.284855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.278303    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:30.284855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.278368    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:30.455214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.453499    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:30.505722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.488393    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:30.505722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.491469    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:30.505722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.492059    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:30.518515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.506820    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:17:30.518515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.508906    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:30.518515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.508960    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:30.627351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.627297    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:30.961053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.952865    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:30.961053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.954515    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:30.995020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:30.994982    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:17:31.120790 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.120526    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:17:31.143372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.142723    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Nov 02 15:17:31.143372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.142793    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Nov 02 15:17:31.232270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.226290    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:31.232270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.226364    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:31.252367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.251451    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:17:31.455212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.454028    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:31.476324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.476207    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 02 15:17:31.476324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.476266    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 02 15:17:31.498800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.495268    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:17:31.508103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.507695    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:17:31.517977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.517860    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:17:31.553623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.553588    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:31.556332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.556308    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:31.559209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.559145    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:17:31.651435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.651387    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:17:31.658244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.658218    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:17:31.658568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:31.658543    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:17:32.230402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:32.230370    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:32.230666 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:32.230629    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:32.573299 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:32.563640    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:32.642552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:32.642521    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:32.680162 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:32.676650    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:17:33.046256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:33.046178    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:33.047736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:33.047713    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:33.051507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:33.051483    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:17:33.221985 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:33.221952    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:33.222168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:33.222001    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:33.604048 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:17:33.603989    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:17:34.222540 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:34.222497    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Nov 02 15:17:34.222947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:34.222569    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Nov 02 15:17:35.227139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:35.227107    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:35.227538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:35.227410    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:35.228077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:35.228061    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:17:38.266219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:38.262738    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 02 15:17:38.404112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:38.403871    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:38.489363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:38.489299    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:38.520522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:38.520343    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:38.522614 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:38.522555    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:17:39.041645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.041584    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:39.118211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.095936    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:39.524220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.507815    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:39.524220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.508616    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:17:39.531360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.531304    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:17:39.550436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.532077    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:17:39.551875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.550711    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:17:39.594868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:39.594839    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:17:40.056830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.056785    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:40.096229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.087306    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:17:40.097679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.097107    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:40.121221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.097927    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:17:40.395098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.394983    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:40.520430 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.509763    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:40.612152 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.536795    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:40.638210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:40.636486    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:41.113822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.113779    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:41.130251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.115733    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:41.403818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.403689    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:41.404982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.404947    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:17:41.745214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.740662    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:17:41.766213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.755714    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:41.823770 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.817450    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:41.823770 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:41.817653    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:17:42.134922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.133460    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:17:42.139720 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.139677    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:42.409274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.406499    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:42.471136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.469667    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:17:42.501435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.501403    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:17:42.509787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.509749    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:17:42.525084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.525058    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:17:42.548552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.548518    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:17:42.568683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:42.568643    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:17:51.364358 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.364321    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:17:51.370028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.370001    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449086 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449035    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449253    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449371    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449453    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449525    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mpnv\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449662    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449760    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449864    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.449933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.449918    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.456211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.456171    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:17:51.551573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551537    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551605    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551637    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551714    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551826    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551901    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.551957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.551941    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552009    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mpnv\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552169    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552230    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552278    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552504 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552479    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552542    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.552648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.552555    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.554438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.554406    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.554534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.554441    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.613027 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.612987    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.613252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.613050    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mpnv\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:51.683664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:51.683632    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:52.125991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.125895    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:17:52.130811 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:17:52.130777    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a378458_a569_41a5_bcc0_98322d53a727.slice/crio-74879a8ca814ce7e67f18f48e8de7b3a0c32d11b71d4910aab1951ff7fcbe843 WatchSource:0}: Error finding container 74879a8ca814ce7e67f18f48e8de7b3a0c32d11b71d4910aab1951ff7fcbe843: Status 404 returned error can't find the container with id 74879a8ca814ce7e67f18f48e8de7b3a0c32d11b71d4910aab1951ff7fcbe843
Nov 02 15:17:52.133377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.133338    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:52.133481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.133409    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:52.133481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.133437    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:17:52.413778 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.413670    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" event={"ID":"8a378458-a569-41a5-bcc0-98322d53a727","Type":"ContainerStarted","Data":"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"}
Nov 02 15:17:52.413778 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.413748    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" event={"ID":"8a378458-a569-41a5-bcc0-98322d53a727","Type":"ContainerStarted","Data":"74879a8ca814ce7e67f18f48e8de7b3a0c32d11b71d4910aab1951ff7fcbe843"}
Nov 02 15:17:52.540318 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.540253    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podStartSLOduration=1.540231023 podStartE2EDuration="1.540231023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:17:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:17:52.53422631 +0000 UTC m=+1278.972128450" watchObservedRunningTime="2026-11-02 15:17:52.540231023 +0000 UTC m=+1278.978133165"
Nov 02 15:17:52.623096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.623057    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:17:52.623400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.623380    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:17:52.623612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.623574    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" containerID="cri-o://a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68" gracePeriod=30
Nov 02 15:17:52.623846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.623824    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy" containerID="cri-o://314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5" gracePeriod=30
Nov 02 15:17:52.690242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:52.684544    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:53.686103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:53.686065    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:53.686950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:53.686912    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:54.692215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:54.691386    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:54.692215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:54.691452    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:55.686676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:55.686632    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:55.687009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:55.686976    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:56.691214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:56.690357    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:56.691214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:56.690466    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:57.694218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:57.687637    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:57.694218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:57.687704    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:57.952878 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:57.952852    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:17:58.107708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107674    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.107905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107721    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.107905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107795    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.107905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107829    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.107905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107854    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.108119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.107911    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.108119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108056    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.108119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108092    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.108285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108139    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:17:58.108455 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108385    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\") pod \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\" (UID: \"14f04132-bc3d-495d-91e3-305a4fb8cf9c\") "
Nov 02 15:17:58.108548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108529    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.108956 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108851    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.108956 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.108922    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.109395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.109369    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.109495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.109402    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.109495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.109419    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.109495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.109442    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.109853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.109514    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.111554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.111471    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:17:58.112616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.112588    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.119688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.119658    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:17:58.119803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.119661    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd" pod "14f04132-bc3d-495d-91e3-305a4fb8cf9c" (UID: "14f04132-bc3d-495d-91e3-305a4fb8cf9c"). InnerVolumeSpecName "kube-api-access-9jlpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:17:58.211106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.210989    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.211106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.211027    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jlpd\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-kube-api-access-9jlpd\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.211106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.211046    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.211106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.211063    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14f04132-bc3d-495d-91e3-305a4fb8cf9c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.211106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.211078    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14f04132-bc3d-495d-91e3-305a4fb8cf9c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.355768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.355739    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:17:58.413714 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.413677    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.413920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.413758    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.413920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.413812    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.413920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.413908    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.413957    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414033    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414043    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.414228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414125    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414173    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.414314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414265    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414328    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\") pod \"a5792826-79f4-4ceb-816d-6d717cb4f970\" (UID: \"a5792826-79f4-4ceb-816d-6d717cb4f970\") "
Nov 02 15:17:58.414376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414362    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.414508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414485    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.414790 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.414766    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:17:58.415280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.415259    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.415369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.415287    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.415369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.415305    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5792826-79f4-4ceb-816d-6d717cb4f970-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.415369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.415321    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.415369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.415335    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.416472 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.416451    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "kube-api-access-8tfcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:17:58.416561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.416520    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:17:58.416772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.416753    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:17:58.416881 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.416864    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy" pod "a5792826-79f4-4ceb-816d-6d717cb4f970" (UID: "a5792826-79f4-4ceb-816d-6d717cb4f970"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:17:58.457089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.457053    4470 generic.go:356] "Generic (PLEG): container finished" podID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerID="a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68" exitCode=0
Nov 02 15:17:58.457327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.457116    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"
Nov 02 15:17:58.457327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.457148    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" event={"ID":"14f04132-bc3d-495d-91e3-305a4fb8cf9c","Type":"ContainerDied","Data":"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"}
Nov 02 15:17:58.457327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.457250    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" event={"ID":"14f04132-bc3d-495d-91e3-305a4fb8cf9c","Type":"ContainerDied","Data":"03b667ab7327e5d02bb2d71e09bbca1fc427aea830314f80b71eef2e96a5f214"}
Nov 02 15:17:58.457327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.457283    4470 scope.go:122] "RemoveContainer" containerID="a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"
Nov 02 15:17:58.459113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.459065    4470 generic.go:356] "Generic (PLEG): container finished" podID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerID="314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5" exitCode=0
Nov 02 15:17:58.459302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.459124    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"
Nov 02 15:17:58.459302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.459145    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" event={"ID":"a5792826-79f4-4ceb-816d-6d717cb4f970","Type":"ContainerDied","Data":"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"}
Nov 02 15:17:58.459302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.459234    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn" event={"ID":"a5792826-79f4-4ceb-816d-6d717cb4f970","Type":"ContainerDied","Data":"ebb972750d7b2227fe69585c25e2f245aff85e24a2fef8331ebcbc17b1870445"}
Nov 02 15:17:58.471118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.471080    4470 scope.go:122] "RemoveContainer" containerID="a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"
Nov 02 15:17:58.471726 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:17:58.471695    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68\": container with ID starting with a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68 not found: ID does not exist" containerID="a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"
Nov 02 15:17:58.471823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.471740    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68"} err="failed to get container status \"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68\": rpc error: code = NotFound desc = could not find container \"a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68\": container with ID starting with a8877accf7cf9dcb550a925086dad03e8a535f2a74db04584152757086e7ff68 not found: ID does not exist"
Nov 02 15:17:58.471823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.471763    4470 scope.go:122] "RemoveContainer" containerID="314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"
Nov 02 15:17:58.505394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.505372    4470 scope.go:122] "RemoveContainer" containerID="314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"
Nov 02 15:17:58.505827 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:17:58.505803    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5\": container with ID starting with 314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5 not found: ID does not exist" containerID="314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"
Nov 02 15:17:58.505896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.505839    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5"} err="failed to get container status \"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5\": rpc error: code = NotFound desc = could not find container \"314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5\": container with ID starting with 314ed52cd4d5aa8c959be98ff04f9c6af50f8d791e225b04bfbcf4ab59b79ae5 not found: ID does not exist"
Nov 02 15:17:58.517065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.517023    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.517065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.517067    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tfcd\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-kube-api-access-8tfcd\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.517313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.517085    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.517313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.517100    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5792826-79f4-4ceb-816d-6d717cb4f970-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:17:58.615405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.615369    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:17:58.628229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.628172    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-mzmxn"]
Nov 02 15:17:58.684719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.684682    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Nov 02 15:17:58.684918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.684766    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Nov 02 15:17:58.731006 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.730907    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:17:58.739195 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.739148    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2"]
Nov 02 15:17:58.910175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.910133    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-f9td2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:17:58.910430 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:58.910233    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-f9td2" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:17:59.688809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:59.688772    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:59.689202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:59.689163    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:59.690036 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:59.690016    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:17:59.713329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:59.709891    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" path="/var/lib/kubelet/pods/14f04132-bc3d-495d-91e3-305a4fb8cf9c/volumes"
Nov 02 15:17:59.713329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:17:59.712779    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" path="/var/lib/kubelet/pods/a5792826-79f4-4ceb-816d-6d717cb4f970/volumes"
Nov 02 15:18:00.413767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.413736    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6761-1a3f0d"]
Nov 02 15:18:00.414414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414397    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy"
Nov 02 15:18:00.414470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414414    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy"
Nov 02 15:18:00.414470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414438    4470 cpu_manager.go:414] "containerMap: removing container" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy"
Nov 02 15:18:00.414470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414443    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy"
Nov 02 15:18:00.414569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414560    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5792826-79f4-4ceb-816d-6d717cb4f970" containerName="istio-proxy"
Nov 02 15:18:00.414600 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.414572    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="14f04132-bc3d-495d-91e3-305a4fb8cf9c" containerName="istio-proxy"
Nov 02 15:18:00.419557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.419535    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:00.452117 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.452087    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k5hh\" (UniqueName: \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\") pod \"test-curl-6761-1a3f0d\" (UID: \"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696\") " pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:00.471612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.471573    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xkc9h\""
Nov 02 15:18:00.479798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.479766    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6761-1a3f0d"]
Nov 02 15:18:00.554403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.554361    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k5hh\" (UniqueName: \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\") pod \"test-curl-6761-1a3f0d\" (UID: \"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696\") " pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:00.615874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.615840    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k5hh\" (UniqueName: \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\") pod \"test-curl-6761-1a3f0d\" (UID: \"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696\") " pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:00.732277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:00.732243    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:01.114316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:01.114287    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6761-1a3f0d"]
Nov 02 15:18:01.116056 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:01.116012    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ee478ec_e5c4_4c4c_9cc3_f1e4b114f696.slice/crio-e5e58a77a853322373cb093801109e7d40fc3011ab6d342ecfbbcb6b3f911ee9 WatchSource:0}: Error finding container e5e58a77a853322373cb093801109e7d40fc3011ab6d342ecfbbcb6b3f911ee9: Status 404 returned error can't find the container with id e5e58a77a853322373cb093801109e7d40fc3011ab6d342ecfbbcb6b3f911ee9
Nov 02 15:18:01.492089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:01.492038    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6761-1a3f0d" event={"ID":"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696","Type":"ContainerStarted","Data":"e5e58a77a853322373cb093801109e7d40fc3011ab6d342ecfbbcb6b3f911ee9"}
Nov 02 15:18:02.500307 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:02.500256    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6761-1a3f0d" event={"ID":"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696","Type":"ContainerStarted","Data":"965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6"}
Nov 02 15:18:02.618764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:02.618703    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6761-1a3f0d" podStartSLOduration=1.638503861 podStartE2EDuration="2.61868723s" totalImagesPullingTime="980.183369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:02.615631539 +0000 UTC m=+1289.053533681" watchObservedRunningTime="2026-11-02 15:18:02.61868723 +0000 UTC m=+1289.056589371"
Nov 02 15:18:03.568291 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:03.568250    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:18:04.066885 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.066843    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6761-1a3f0d"]
Nov 02 15:18:04.070769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.070733    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6761-1a3f0d"]
Nov 02 15:18:04.227462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.227427    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6761-77254d"]
Nov 02 15:18:04.228045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.228030    4470 cpu_manager.go:414] "containerMap: removing container" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" containerName="test-curl-6761-1a3f0d"
Nov 02 15:18:04.228107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.228046    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" containerName="test-curl-6761-1a3f0d"
Nov 02 15:18:04.228179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.228169    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" containerName="test-curl-6761-1a3f0d"
Nov 02 15:18:04.231782 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.231766    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:04.287765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.287727    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6761-77254d"]
Nov 02 15:18:04.310756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.310713    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94c48\" (UniqueName: \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\") pod \"test-curl-6761-77254d\" (UID: \"99542563-fa47-45f7-909b-01f0b7064a47\") " pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:04.412224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.412111    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94c48\" (UniqueName: \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\") pod \"test-curl-6761-77254d\" (UID: \"99542563-fa47-45f7-909b-01f0b7064a47\") " pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:04.473331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.473298    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94c48\" (UniqueName: \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\") pod \"test-curl-6761-77254d\" (UID: \"99542563-fa47-45f7-909b-01f0b7064a47\") " pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:04.554860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.554820    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:04.565211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.561779    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6761-1a3f0d" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" containerName="test-curl-6761-1a3f0d" containerID="cri-o://965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6" gracePeriod=2
Nov 02 15:18:04.613057 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:04.613012    4470 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6761-1a3f0d\" is forbidden: User \"system:node:ip-10-0-139-197.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-197.us-west-1.compute.internal' and this object" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:04.975155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:04.975117    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6761-77254d"]
Nov 02 15:18:04.977170 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:04.977142    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99542563_fa47_45f7_909b_01f0b7064a47.slice/crio-6b2d78c6ecd0ba98ca533c8e0413413e45970f2424a57bbb677d9d46c1da76e7 WatchSource:0}: Error finding container 6b2d78c6ecd0ba98ca533c8e0413413e45970f2424a57bbb677d9d46c1da76e7: Status 404 returned error can't find the container with id 6b2d78c6ecd0ba98ca533c8e0413413e45970f2424a57bbb677d9d46c1da76e7
Nov 02 15:18:05.571666 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.571634    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6761-77254d" event={"ID":"99542563-fa47-45f7-909b-01f0b7064a47","Type":"ContainerStarted","Data":"c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789"}
Nov 02 15:18:05.571844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.571677    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6761-77254d" event={"ID":"99542563-fa47-45f7-909b-01f0b7064a47","Type":"ContainerStarted","Data":"6b2d78c6ecd0ba98ca533c8e0413413e45970f2424a57bbb677d9d46c1da76e7"}
Nov 02 15:18:05.685522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.685466    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6761-77254d" podStartSLOduration=1.685453007 podStartE2EDuration="1.685453007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:05.68505785 +0000 UTC m=+1292.122959990" watchObservedRunningTime="2026-11-02 15:18:05.685453007 +0000 UTC m=+1292.123355162"
Nov 02 15:18:05.735936 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:05.735889    4470 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6761-1a3f0d\" is forbidden: User \"system:node:ip-10-0-139-197.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-197.us-west-1.compute.internal' and this object" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:05.841492 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.841401    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:18:05.846689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.846665    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.938319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.938279    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:18:05.938673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.938536    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.938879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.938780    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrwvv\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.938881    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.938944    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.939009    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.939035    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.939138    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.939243    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:05.939722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:05.939349    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040276    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040424    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040494    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040563    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040654    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrwvv\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040704    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040734    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040764    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040784    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.040954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.040837    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.041118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.041087    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.041242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.041164    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.041332 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.041309    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.041401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.041375    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.042793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.042773    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.043033 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.043013    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.104758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.104664    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.104929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.104792    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrwvv\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\") pod \"e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") " pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.162301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.162269    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:06.607117 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.607087    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:18:06.608555 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:06.608532    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c60443_8c58_466a_ae68_95b76e45de67.slice/crio-eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8 WatchSource:0}: Error finding container eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8: Status 404 returned error can't find the container with id eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8
Nov 02 15:18:06.649879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.649825    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:06.650062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.649926    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:06.650062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.649973    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:06.755835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.755793    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:18:06.763524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.763493    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.782643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.782608    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:18:06.787289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.787266    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.823080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.823044    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:18:06.855665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855568    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.855665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855652    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.855902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855681    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.855902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855737    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.855902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855767    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.855902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855861    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.855902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855887    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855916    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.855959    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.856153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856023    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856096    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856135    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.856588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856210    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.856588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856317    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcfdc\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856375    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.856588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856407    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqrh\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.856588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856439    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.856927 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.856621    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.874051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.873227    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:18:06.903027 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.902994    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:06.958153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958110    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958165    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958233    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958269    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958326    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958364    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958398    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958432    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958488    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcfdc\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958522    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958541    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqrh\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958560    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958636    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.958727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958712    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958759    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.959056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958777    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958811    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.958847    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.959056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959016    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.959281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959146    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959224    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.959389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959288    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.959547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959523    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959654    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.959865 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.959836    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.960134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.960109    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.960320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.960296    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.960791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.960763    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.961494 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.961471    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.961764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.961740    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:06.962607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.962584    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:06.962866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:06.962846    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:07.021369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.021336    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:07.021553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.021532    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:07.021599 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.021535    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqrh\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:07.021911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.021892    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcfdc\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:07.059756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.059723    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\" (UniqueName: \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\") pod \"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696\" (UID: \"0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696\") "
Nov 02 15:18:07.061864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.061833    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh" pod "0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" (UID: "0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696"). InnerVolumeSpecName "kube-api-access-6k5hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:18:07.085867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.085836    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:07.155149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.155110    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:07.162756 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.162221    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k5hh\" (UniqueName: \"kubernetes.io/projected/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696-kube-api-access-6k5hh\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:18:07.249140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.249104    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6761-77254d"]
Nov 02 15:18:07.257243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.256473    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6761-77254d"]
Nov 02 15:18:07.489390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.486716    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:18:07.495607 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:07.495567    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc7a91a_d2ce_4f6c_a932_0ecf39f185a4.slice/crio-dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0 WatchSource:0}: Error finding container dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0: Status 404 returned error can't find the container with id dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0
Nov 02 15:18:07.498929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.498887    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.499024 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.498982    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.499097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.499020    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.576271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.576238    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:18:07.580610 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:07.580583    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb4dff8_15f7_4c63_9bef_2ef9b7cbcdc3.slice/crio-62d86380dfb1a20941f227652470a5c08ee2b2d8d40eb89e22bee3465d589ad6 WatchSource:0}: Error finding container 62d86380dfb1a20941f227652470a5c08ee2b2d8d40eb89e22bee3465d589ad6: Status 404 returned error can't find the container with id 62d86380dfb1a20941f227652470a5c08ee2b2d8d40eb89e22bee3465d589ad6
Nov 02 15:18:07.593709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.593658    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.593840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.593771    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.593840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.593819    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:07.606049 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.606012    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" event={"ID":"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4","Type":"ContainerStarted","Data":"1e81b4a0a95fd3d21164e1ba94e40bd39576be5c3870587822ae2a26f09ff662"}
Nov 02 15:18:07.606223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.606076    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" event={"ID":"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4","Type":"ContainerStarted","Data":"dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0"}
Nov 02 15:18:07.610067 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.610027    4470 generic.go:356] "Generic (PLEG): container finished" podID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" containerID="965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6" exitCode=137
Nov 02 15:18:07.610573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.610143    4470 scope.go:122] "RemoveContainer" containerID="965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6"
Nov 02 15:18:07.610573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.610148    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-1a3f0d"
Nov 02 15:18:07.613950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.613753    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6761-77254d" podUID="99542563-fa47-45f7-909b-01f0b7064a47" containerName="test-curl-6761-77254d" containerID="cri-o://c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789" gracePeriod=2
Nov 02 15:18:07.613950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.613912    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" event={"ID":"81c60443-8c58-466a-ae68-95b76e45de67","Type":"ContainerStarted","Data":"190efd00b2fff32c1cf328bd363fb6e0b0aca6baf6912941c5c262124d2bbf90"}
Nov 02 15:18:07.614121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.613967    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" event={"ID":"81c60443-8c58-466a-ae68-95b76e45de67","Type":"ContainerStarted","Data":"eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8"}
Nov 02 15:18:07.639816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.639786    4470 scope.go:122] "RemoveContainer" containerID="965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6"
Nov 02 15:18:07.640329 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:07.640300    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6\": container with ID starting with 965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6 not found: ID does not exist" containerID="965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6"
Nov 02 15:18:07.640434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.640341    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6"} err="failed to get container status \"965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6\": rpc error: code = NotFound desc = could not find container \"965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6\": container with ID starting with 965b947923fc186fdd8a69938eb6dd54ba360c65604d66040e8681a65d0f33e6 not found: ID does not exist"
Nov 02 15:18:07.726428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.726398    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696" path="/var/lib/kubelet/pods/0ee478ec-e5c4-4c4c-9cc3-f1e4b114f696/volumes"
Nov 02 15:18:07.736398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.736301    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podStartSLOduration=1.73627959 podStartE2EDuration="1.73627959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:07.729885529 +0000 UTC m=+1294.167787674" watchObservedRunningTime="2026-11-02 15:18:07.73627959 +0000 UTC m=+1294.174181731"
Nov 02 15:18:07.902149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:07.893274    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podStartSLOduration=2.89324868 podStartE2EDuration="2.89324868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:07.866552557 +0000 UTC m=+1294.304454699" watchObservedRunningTime="2026-11-02 15:18:07.89324868 +0000 UTC m=+1294.331150822"
Nov 02 15:18:08.093570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.093465    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:08.102224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.096795    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:08.102224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.096853    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:08.162990 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.162958    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:08.164949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.164918    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:08.165146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.164981    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:08.648215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.646634    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" event={"ID":"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3","Type":"ContainerStarted","Data":"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"}
Nov 02 15:18:08.648215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:08.646746    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" event={"ID":"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3","Type":"ContainerStarted","Data":"62d86380dfb1a20941f227652470a5c08ee2b2d8d40eb89e22bee3465d589ad6"}
Nov 02 15:18:09.096214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.095554    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:09.096214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.095618    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:09.157356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.157316    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:09.157643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.157601    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:09.157766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.157651    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:09.162954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.162926    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:09.163078 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:09.162969    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:10.128949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.128917    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:10.129493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.129459    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:10.158341 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.158296    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:10.158509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.158365    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:10.163458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.163424    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:10.163600 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.163471    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:10.768613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.768528    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:10.864899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.864865    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\" (UniqueName: \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\") pod \"99542563-fa47-45f7-909b-01f0b7064a47\" (UID: \"99542563-fa47-45f7-909b-01f0b7064a47\") "
Nov 02 15:18:10.922980 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.919025    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48" pod "99542563-fa47-45f7-909b-01f0b7064a47" (UID: "99542563-fa47-45f7-909b-01f0b7064a47"). InnerVolumeSpecName "kube-api-access-94c48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:18:10.989216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:10.981040    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94c48\" (UniqueName: \"kubernetes.io/projected/99542563-fa47-45f7-909b-01f0b7064a47-kube-api-access-94c48\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:18:11.045267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.045044    4470 generic.go:356] "Generic (PLEG): container finished" podID="99542563-fa47-45f7-909b-01f0b7064a47" containerID="c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789" exitCode=137
Nov 02 15:18:11.045267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.045247    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:11.046301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.046269    4470 scope.go:122] "RemoveContainer" containerID="c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789"
Nov 02 15:18:11.105283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.091226    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:11.105283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.091464    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:11.158561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.156376    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:11.158561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.156433    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:11.175825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.170777    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:11.175825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.170834    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:11.259217 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:11.244880    4470 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6761-77254d\" is forbidden: User \"system:node:ip-10-0-139-197.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-197.us-west-1.compute.internal' and this object" podUID="99542563-fa47-45f7-909b-01f0b7064a47" pod="opendatahub/test-curl-6761-77254d"
Nov 02 15:18:11.259217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.252379    4470 scope.go:122] "RemoveContainer" containerID="c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789"
Nov 02 15:18:11.259217 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:11.252968    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789\": container with ID starting with c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789 not found: ID does not exist" containerID="c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789"
Nov 02 15:18:11.259217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.253010    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789"} err="failed to get container status \"c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789\": rpc error: code = NotFound desc = could not find container \"c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789\": container with ID starting with c436b5178c068739aed6e3bfeeb4f9a62e1c7a6d2493752a836917becacfd789 not found: ID does not exist"
Nov 02 15:18:11.800222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:11.794472    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99542563-fa47-45f7-909b-01f0b7064a47" path="/var/lib/kubelet/pods/99542563-fa47-45f7-909b-01f0b7064a47/volumes"
Nov 02 15:18:12.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.032538    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podStartSLOduration=6.032515011 podStartE2EDuration="6.032515011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:08.851937534 +0000 UTC m=+1295.289839676" watchObservedRunningTime="2026-11-02 15:18:12.032515011 +0000 UTC m=+1298.470417154"
Nov 02 15:18:12.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.033717    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:18:12.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.036010    4470 cpu_manager.go:414] "containerMap: removing container" podUID="99542563-fa47-45f7-909b-01f0b7064a47" containerName="test-curl-6761-77254d"
Nov 02 15:18:12.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.036031    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99542563-fa47-45f7-909b-01f0b7064a47" containerName="test-curl-6761-77254d"
Nov 02 15:18:12.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.036252    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="99542563-fa47-45f7-909b-01f0b7064a47" containerName="test-curl-6761-77254d"
Nov 02 15:18:12.075268 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.066996    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.162288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.154852    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:12.162288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.154906    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:12.162288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.161057    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:12.162288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.161107    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:12.184214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.171554    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:12.184214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.171781    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:12.195216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.193218    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-ad0b4b5e-openshift-default-dockercfg-4zvlm\""
Nov 02 15:18:12.208297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.201216    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.208297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.201264    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.208297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.201420    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.227457    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqmk7\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.227568    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.227675    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.227933    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.228020    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.237209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.228043    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.287465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.287335    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360134    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360269    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360300    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360344    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360369    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360415    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360452    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqmk7\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360530    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.360628    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.361234    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.361556    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.361786    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.378735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.362438    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.417444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.417333    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.513217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.512445    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.513217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.513146    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.707757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.592828    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:12.953057 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:12.952980    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqmk7\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\") pod \"e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") " pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:13.123316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.091260    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:13.123316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.091337    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:13.126155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.124684    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.146358    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.148557    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.160423    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.160485    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.169945    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:13.171212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.169998    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:13.448217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.444416    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:13.546214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.527295    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:13.546214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.527401    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:13.691352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.639591    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:13.775214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:13.762503    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:14.149391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.149279    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:14.158111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.150847    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:14.158111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.150899    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:14.158111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.152320    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:18:14.174851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.170838    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:14.174851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.172614    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:18:14.489225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.488885    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:14.489225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.488960    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:14.520215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.324171    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:14.520215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.490886    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:14.544220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.537689    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:14.549881 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.549837    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:14.601086 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.601043    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:14.659070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:14.659036    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:18:15.114101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.114053    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:15.116235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.116174    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:15.135272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.132879    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:18:15.135272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.132957    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:18:15.235231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.228111    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:15.235231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.228164    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:15.251425 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.251377    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:15.264415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.264355    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:15.280812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.280773    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:15.281094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.281043    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:15.281871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.281842    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7" podUID="1da922da-c4c3-4c35-9edb-45f6bd072094" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:15.343217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.308492    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:15.361214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.353219    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:18:15.547079 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.546301    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:15.621217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.614418    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:15.797278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.783501    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Nov 02 15:18:15.859287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.814315    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:16.063809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.063701    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.064070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:15.986422    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.123214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.115059    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.123553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.123513    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.146209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.146149    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:16.146513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.146482    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:16.360295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.345648    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:16.360295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.345706    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:16.404740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.404635    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.404934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.404901    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:18:16.549216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.428368    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:18:16.677276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.665443    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.686360    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.703381    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.703449    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.705731    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.708552    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:16.768278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.708602    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:16.894672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.894628    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:16.894936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.894904    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.971378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.971331    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:16.975254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.971599    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:16.994323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:16.974388    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:17.033628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.033571    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:17.033954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.033926    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:17.215274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.163566    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:17.215274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.163639    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:17.215274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.190347    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.230270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.208565    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.241383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.241344    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:17.284310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.284264    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:17.297429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.267333    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:17.305829 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.301261    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:17.492851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.492748    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:17.495159 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.492942    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.556224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.552290    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:17.556224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.552364    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.727869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.727816    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.729477 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.729353    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:17.729477 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.729419    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:17.729701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:17.729668    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:18.143667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.143633    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:18.164737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.164523    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:18.165008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.164967    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:18.165155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.164562    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:18.165300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.165280    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:18.177310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.166228    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:18.243316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.193548    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:18.265132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.245668    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:18.272667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.272613    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 02 15:18:18.520934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.520883    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:18.537637 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.537593    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:18.730309 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.730262    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:18.841502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.841405    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:18.934552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.934512    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:18.948317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:18.948267    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.084296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.084231    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerStarted","Data":"45f69be937954165b56a0b0ac2e1304308923d1f9633211c0dc3a775898fc9c4"}
Nov 02 15:18:19.206249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.202434    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:19.243164 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.243128    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:19.243425 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.243063    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:19.529212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.495781    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:19.529212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.495854    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:19.568216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.531263    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.595283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.573982    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:19.595283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.574062    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:19.792272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.784317    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.956215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.804850    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:19.956215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.821399    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.956215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.923786    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.956215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.924254    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:19.983258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:19.978274    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:20.187508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.128139    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:20.187508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.159643    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:20.187508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.159708    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:20.257220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.257092    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:20.416406 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.416350    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:20.420348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.420268    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:20.512374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.459721    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:20.543364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.543275    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:20.860423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.860302    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:21.054217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.034243    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:21.054489 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:20.960438    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:21.113727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.113628    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:21.113974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.113940    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:21.203993 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:21.203956    4470 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.484s"
Nov 02 15:18:21.247220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.223948    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:21.247220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.224017    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:21.247220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.232269    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:21.247220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.236062    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:21.282760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.282712    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:21.283013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.282982    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:21.311896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.311851    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/echo-simulator-kserve-64b6f8f96c-fxkf2" podUID="b51259a2-7ada-4385-82b0-3586cfdb59ba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:21.317655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.317617    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerStarted","Data":"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"}
Nov 02 15:18:21.365644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.365548    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:21.373342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.373288    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:21.504619 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.504585    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:18:21.583219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.546277    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.699724 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.699689    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c82fc808-gw-openshift-default-dockercfg-z925c\""
Nov 02 15:18:21.743529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.743484    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.744764    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.902422    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904254    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904295    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904392    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904443    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904523    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904570    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmbbp\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.904632    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:21.926228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.905644    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/health\": context deadline exceeded"
Nov 02 15:18:21.935111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:21.744839    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:22.086577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.077423    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:22.103615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.087376    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:22.103615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.088356    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:22.103615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.088662    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.105862    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.105924    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106008    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106033    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106113    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106162    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106250    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.106291    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmbbp\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.116514    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.116588    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.251539    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.260166    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:22.322286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.260538    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:22.432213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.376352    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.432213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.412470    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.432532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.432253    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:18:22.432702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.432668    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.447343 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.437363    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.455229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.450081    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.535920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.535876    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.661554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.661442    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podStartSLOduration=11.661419269 podStartE2EDuration="11.661419269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:21.949290846 +0000 UTC m=+1308.387192988" watchObservedRunningTime="2026-11-02 15:18:22.661419269 +0000 UTC m=+1309.099321424"
Nov 02 15:18:22.794225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.771379    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.794225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.758786    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.794225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.764259    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmbbp\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.794225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.766135    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\") pod \"e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") " pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.794225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.793488    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.719772    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.798827    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.726309    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.798889    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.799614    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:18:22.820768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.819492    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:22.870294 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.870262    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:22.904810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.904735    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:18:22.948757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:22.946351    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:23.080099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.080048    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:23.095147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.094291    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:23.096026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.095409    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:23.109959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.109925    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:23.110288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.110244    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:23.187376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.172797    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:23.187743 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.187694    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:23.207414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.207266    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:23.207414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.207326    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:23.303038 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.302333    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:23.390880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.386525    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:18:23.414286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.414236    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:23.533917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.533876    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:18:23.849265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.807287    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:23.961223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:23.947080    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:24.194318 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.096039    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:24.292344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.292291    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:24.344856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.324613    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:18:24.344856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.325558    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:24.345370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.345324    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:24.405213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.380577    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:24.622458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.622148    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:24.622862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.388777    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Nov 02 15:18:24.623034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.623003    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Nov 02 15:18:24.653645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.653596    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:18:24.653967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.653949    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:24.655478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.655436    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:24.668213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.665973    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:24.699225 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:24.699076    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d5d7e35_6e0c_4639_bad6_3a3752ba3c53.slice/crio-318db947b6628258b6fce31b963e45b6b3d1125d1d4d3ab5aad6f2594713cc40 WatchSource:0}: Error finding container 318db947b6628258b6fce31b963e45b6b3d1125d1d4d3ab5aad6f2594713cc40: Status 404 returned error can't find the container with id 318db947b6628258b6fce31b963e45b6b3d1125d1d4d3ab5aad6f2594713cc40
Nov 02 15:18:24.735078 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.735038    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:18:24.770223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.768773    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:24.770223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.768881    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:24.770223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:24.768929    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:25.033469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.033416    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerStarted","Data":"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"}
Nov 02 15:18:25.033893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.033497    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerStarted","Data":"318db947b6628258b6fce31b963e45b6b3d1125d1d4d3ab5aad6f2594713cc40"}
Nov 02 15:18:25.087466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.087417    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Nov 02 15:18:25.087658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.087478    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Nov 02 15:18:25.177260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.155858    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 02 15:18:25.177260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.155952    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 02 15:18:25.177260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.169632    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podStartSLOduration=6.169611654 podStartE2EDuration="6.169611654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:25.166860806 +0000 UTC m=+1311.604762949" watchObservedRunningTime="2026-11-02 15:18:25.169611654 +0000 UTC m=+1311.607513795"
Nov 02 15:18:25.826303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.826270    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:25.834687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:25.834660    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:26.132401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.130005    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:26.138032 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.137538    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:18:26.170705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.170329    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:26.170705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.170397    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:26.877479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.877444    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:26.883322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.883288    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:18:26.958484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.958449    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:26.959987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:26.959964    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:18:27.213972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:27.213934    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:27.214568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:27.214532    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:27.889215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:27.886860    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:27.892344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:27.889718    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:18:33.565834 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:33.565793    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:18:36.731289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.731254    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:18:36.737034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.737012    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.853633 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.853354    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:18:36.863934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.863870    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjlwp\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.863998    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864090    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864163    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864410    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864450    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864501    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864536    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.864662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.864619    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965712    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965758    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965788    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965812    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965861    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965917    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjlwp\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.965978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.965970    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966060    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966126    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966275    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966393    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966494 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966448    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966492    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.966627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.966605    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.968577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.968555    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.968824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.968804    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:36.983135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.983058    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:18:36.989901 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.989878    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:18:36.990039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.989987    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:36.994309 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:36.994288    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.026924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.026891    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:37.027084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.026942    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjlwp\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\") pod \"e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") " pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:37.051130 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.050913    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:37.062073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.062047    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:18:37.064932 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.064877    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:18:37.069282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.069255    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.069651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.069629    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqmjg\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.069815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.069800    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.069958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.069918    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.070182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070156    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.070295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070229    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.070295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070266    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.070402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070293    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070771    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070813    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070870    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070919    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070947    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.070981    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.071004    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.071058    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m67mf\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.071137    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.071689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.071233    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173196 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173151    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m67mf\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173237    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173308    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173386    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173434    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqmjg\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173498    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173539    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173605    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.173692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173651    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173685    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173712    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173810    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173848    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173912    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.173988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173966    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.174061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.173991    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.174061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.174031    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.174181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.174058    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.175264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.175238    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.175838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.175787    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.176046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.176019    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.177376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.177342    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.178149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.177841    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.178149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.178130    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.178493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.178404    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.179549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.178668    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.179549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.178822    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.179549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.179044    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.179549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.179155    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.182078 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.180154    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.182078 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.181801    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.182936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.182909    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.241003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.240930    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.241247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.241229    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m67mf\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\") pod \"e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") " pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.244721 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.244698    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqmjg\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.244992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.244973    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\") pod \"e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") " pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.309079 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.309047    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:37.315954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.315929    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:37.511055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.511029    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:18:37.511435 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:37.511415    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca3b944_22d2_4b34_921f_eccfb3b1090c.slice/crio-d4ede90d76e09418587be982306885fbb4f23873d7576e847e0f1197832048dd WatchSource:0}: Error finding container d4ede90d76e09418587be982306885fbb4f23873d7576e847e0f1197832048dd: Status 404 returned error can't find the container with id d4ede90d76e09418587be982306885fbb4f23873d7576e847e0f1197832048dd
Nov 02 15:18:37.514567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.514525    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.514647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.514621    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.514686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.514673    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.708412 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:37.708384    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cffb1ec_3c2e_4c1b_a502_938b3b1a0d17.slice/crio-d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5 WatchSource:0}: Error finding container d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5: Status 404 returned error can't find the container with id d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5
Nov 02 15:18:37.708681 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.708661    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:18:37.710961 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.710934    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.711026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.710999    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.711069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.711027    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.760156 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.760089    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:18:37.766849 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:18:37.766814    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad75510_0fdf_4413_9225_87e9b3d6bd54.slice/crio-19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27 WatchSource:0}: Error finding container 19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27: Status 404 returned error can't find the container with id 19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27
Nov 02 15:18:37.769971 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.769925    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.770049 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.770026    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:37.770122 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:37.770101    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:38.192263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.174264    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:18:38.192263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.179903    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.192263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.185871    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373399    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373481    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373511    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373538    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nksds\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373606    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373683    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373795    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373841    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.376213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.373895    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475339    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475437    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475542    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475586    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475637    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475878    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475935    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475961    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.477207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.475985    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nksds\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.510228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.486383    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.510228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.487282    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.519579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.519541    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.520356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.520327    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.521108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.520368    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.521324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.520620    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.522779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.522752    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" event={"ID":"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17","Type":"ContainerStarted","Data":"a9e84e41e9e3b11836cfbcd059a2ff50af58b4de6c6d369f2cd6112490ed7a7a"}
Nov 02 15:18:38.522870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.522812    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" event={"ID":"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17","Type":"ContainerStarted","Data":"d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5"}
Nov 02 15:18:38.536220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.526238    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" event={"ID":"bca3b944-22d2-4b34-921f-eccfb3b1090c","Type":"ContainerStarted","Data":"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"}
Nov 02 15:18:38.536220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.526299    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" event={"ID":"bca3b944-22d2-4b34-921f-eccfb3b1090c","Type":"ContainerStarted","Data":"d4ede90d76e09418587be982306885fbb4f23873d7576e847e0f1197832048dd"}
Nov 02 15:18:38.536220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.530024    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" event={"ID":"9ad75510-0fdf-4413-9225-87e9b3d6bd54","Type":"ContainerStarted","Data":"bedfc0f48d6bcc7cc5dd6640a87ebe0c8e0051a3f31bd517bda2679e1268fde4"}
Nov 02 15:18:38.536220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.530092    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" event={"ID":"9ad75510-0fdf-4413-9225-87e9b3d6bd54","Type":"ContainerStarted","Data":"19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27"}
Nov 02 15:18:38.727167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.727088    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.782809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.782774    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.783400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.782906    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nksds\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\") pod \"e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") " pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:38.974217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:38.954927    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podStartSLOduration=2.954901897 podStartE2EDuration="2.954901897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:38.819038118 +0000 UTC m=+1325.256940274" watchObservedRunningTime="2026-11-02 15:18:38.954901897 +0000 UTC m=+1325.392804040"
Nov 02 15:18:39.126681 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.126592    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:39.127107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.127072    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:39.127340 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.127307    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:39.318899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.135999    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podStartSLOduration=3.135977387 podStartE2EDuration="3.135977387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:39.002263382 +0000 UTC m=+1325.440165518" watchObservedRunningTime="2026-11-02 15:18:39.135977387 +0000 UTC m=+1325.573879530"
Nov 02 15:18:39.328013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.189363    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:39.340454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.269332    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:39.340885 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.311683    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:39.341009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.340900    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:39.341009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.311752    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:39.341009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.340984    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:39.341214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.338046    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:39.341214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.341069    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:39.444217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.438751    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:39.548513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:39.548450    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podStartSLOduration=3.548428263 podStartE2EDuration="3.548428263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:39.394877781 +0000 UTC m=+1325.832779914" watchObservedRunningTime="2026-11-02 15:18:39.548428263 +0000 UTC m=+1325.986330405"
Nov 02 15:18:40.065051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.065011    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:40.065580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.065086    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:40.337627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.337530    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:40.337906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.337874    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:40.349325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.349120    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:40.349325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.349240    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:40.361524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.361489    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:18:40.416216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.407263    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:40.416216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.407372    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:40.416216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.407418    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:18:40.653377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:40.647367    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" event={"ID":"642ce871-6b62-4d7d-9fc9-2df3207a499c","Type":"ContainerStarted","Data":"8f29bd5e08de90795b7e17d858888369dfb546fa63ab203dd3ccd8f2bd12021e"}
Nov 02 15:18:41.131214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.094310    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:41.131214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.094379    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:41.318234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.310251    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:41.318234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.310335    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:41.350214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.343401    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:41.350214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.343478    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:41.953794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:41.953747    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" event={"ID":"642ce871-6b62-4d7d-9fc9-2df3207a499c","Type":"ContainerStarted","Data":"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"}
Nov 02 15:18:42.209317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.199556    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:42.209998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.209947    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:42.225216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.213605    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:42.275867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.275818    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:42.417216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.394599    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:42.495276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.472680    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:42.533905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.533844    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:42.574450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.574402    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:42.586501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.586453    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:42.847497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.846006    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:42.850372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.850319    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:42.856316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.853465    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podStartSLOduration=5.85344213 podStartE2EDuration="5.85344213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:18:42.710828393 +0000 UTC m=+1329.148730568" watchObservedRunningTime="2026-11-02 15:18:42.85344213 +0000 UTC m=+1329.291344273"
Nov 02 15:18:42.861425 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.858766    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:42.862399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.862354    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:42.862652 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:42.862610    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:43.039612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.039572    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:43.039977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.039910    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:43.053940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.053772    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:43.054260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.054223    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:43.316213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.313615    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:43.316213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.313679    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:43.327222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.326693    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:43.327222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:43.326770    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:44.068216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.056466    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:44.068216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.056543    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:44.068216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.056643    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:44.068216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.056668    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:44.306365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.306300    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:44.310288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.310249    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:44.310632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.310597    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:44.319913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.318399    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:44.319913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.318459    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:44.361211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.353427    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:44.447414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.447359    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:44.717212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:44.714345    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:45.058013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.057915    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:45.058338 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.058293    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:45.058519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.058282    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:45.060715 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.060688    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:45.060904 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.060881    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:45.257309 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.257267    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:45.336485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.336390    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:45.337734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.337708    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:18:45.342208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.337811    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:45.342208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.339281    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:45.342208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.338095    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:45.342208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.339434    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:45.425211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.419282    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:45.425211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.419799    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:18:45.478866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.456149    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Nov 02 15:18:45.973084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:45.973021    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": context deadline exceeded"
Nov 02 15:18:46.087070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.087021    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:46.087366 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.087331    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:46.087551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.087226    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:46.087699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.087678    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:46.088667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.088640    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:46.094047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.094018    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": context deadline exceeded"
Nov 02 15:18:46.448267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.429635    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:46.448267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.429720    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:18:46.448267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.429775    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:18:46.448267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.439850    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:46.510876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.493747    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:46.510876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.493810    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:46.560096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.530833    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:46.560096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.545888    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:46.658215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.641017    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:47.009214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:46.978037    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:47.057547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.057426    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:47.057547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.057486    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:47.057912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.057888    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:47.058051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.058027    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:47.058524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.058496    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:47.091844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.091801    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:47.318570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.318035    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:47.318570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.318097    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:47.318570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.318432    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:47.318570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.318460    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:47.456591 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.456546    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:47.800223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.799456    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:18:47.810215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.807573    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:18:47.919770 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:47.919728    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:48.060272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.036615    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:48.060272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.036670    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:48.124330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.117522    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Nov 02 15:18:48.124330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.117588    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Nov 02 15:18:48.326327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.326039    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused" start-of-body=
Nov 02 15:18:48.326327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.326107    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.91:15021/healthz/ready\": dial tcp 10.133.0.91:15021: connect: connection refused"
Nov 02 15:18:48.327504 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.327475    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 02 15:18:48.327744 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:48.327716    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 02 15:18:49.036461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.036412    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:49.037096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.037062    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:49.857361 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.857332    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:49.858338 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.858307    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:49.859466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.859442    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:18:49.896675 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.896642    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:49.897373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.897349    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:49.898284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.898263    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:18:49.986209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.986161    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:49.988354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.988307    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:49.989146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:49.989118    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:18:50.037100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:50.037064    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:50.037567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:50.037123    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:51.040520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:51.040477    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused" start-of-body=
Nov 02 15:18:51.072346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:51.072282    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": dial tcp 10.133.0.92:15021: connect: connection refused"
Nov 02 15:18:51.972212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:51.960304    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:52.307272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.256669    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:52.424276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.423812    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:18:52.452219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.444456    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:52.884698 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.883293    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:52.961348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.961292    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:52.986216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:52.963086    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:53.058312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.058267    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:53.104895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.104855    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:53.105182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.105146    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:53.377175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.377124    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:53.398215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.385980    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:18:53.531367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.405711    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:53.531367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.461304    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:53.531367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.480440    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:18:53.531367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.481023    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:18:53.531367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.481634    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:18:53.967216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.914526    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:53.967690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:53.967643    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:54.086214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.074215    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:54.250281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.232303    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:54.250281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.232378    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:54.469279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.467060    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 02 15:18:54.555433 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.543481    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:54.592131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:54.555763    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:54.949420 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:18:54.929913    4470 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.196s"
Nov 02 15:18:55.164328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.019152    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:55.164328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.019202    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:55.164328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.103298    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:55.275336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.275247    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:18:55.362231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.345363    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:18:55.362231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.345406    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:18:55.362231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.314856    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:55.362231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.345894    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:55.473304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.473251    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:55.965432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:55.964945    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/health\": context deadline exceeded"
Nov 02 15:18:56.201600 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.201538    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:56.261289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.261166    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:56.306256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.297812    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:56.359466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.336675    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:56.359466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.341385    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:56.540269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.460450    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/health\": context deadline exceeded"
Nov 02 15:18:56.540269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.461336    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:56.540269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.461401    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:56.661213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.625434    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/health\": context deadline exceeded"
Nov 02 15:18:56.722649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:56.722598    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.229750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.229701    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.296709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.280315    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:57.330209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.298158    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.375247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.336934    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.375247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.348379    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.681282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.654373    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:57.825368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.825302    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.894314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.894239    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.948458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.938755    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:57.974240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.972079    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:57.974240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.949801    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:57.974240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:57.972447    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.066255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.066215    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:58.211274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.210404    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.347268 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.310007    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.381224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.348284    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": context deadline exceeded"
Nov 02 15:18:58.381224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.348558    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 02 15:18:58.402165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.401310    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.431753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.431630    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:18:58.462397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.460779    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:18:58.478320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.478280    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:58.478595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.478562    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.838043 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.837943    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:58.862247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.852321    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:58.942230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:58.940837    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:59.138918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.130323    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:18:59.138918 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.130391    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:59.330216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.329484    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:18:59.356576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.356297    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:59.359594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.359546    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:18:59.437444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.437405    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:18:59.461069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:18:59.461011    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:19:00.051691 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.051629    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:19:00.104247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.073730    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:19:00.104247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.088256    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:19:00.342605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.342494    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 02 15:19:00.362347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.362301    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" probeResult="failure" output="Get \"https://10.133.0.74:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:19:00.387321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.364276    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:19:00.428796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.428762    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:19:00.450311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.450249    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" probeResult="failure" output="Get \"https://10.133.0.66:8000/ready\": context deadline exceeded"
Nov 02 15:19:00.479356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.479310    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" probeResult="failure" output="Get \"https://10.133.0.68:8000/ready\": context deadline exceeded"
Nov 02 15:19:00.631659 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.631567    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:19:00.736226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:00.731709    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:19:01.164896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.164866    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:19:01.188789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.171728    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:19:01.188789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.181844    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:19:01.188789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.181904    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.92:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:19:01.250444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.250408    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:19:01.250654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.250586    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:19:01.712293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.712258    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:19:01.714087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:01.714061    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:19:03.567121 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:03.567085    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:19:16.506588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.506548    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:19:16.512551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.512522    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.567846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.567805    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67qr2\" (UniqueName: \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.568120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.567989    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.568120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.568069    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ad0b4b5e-serving-cert\""
Nov 02 15:19:16.568613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.568582    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.568747 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.568709    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.572622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.572597    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:19:16.670509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.670476    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.670689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.670557    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.670689 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:16.670613    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-serving-cert: secret "maas-api-e2e-ait-ad0b4b5e-serving-cert" not found
Nov 02 15:19:16.670689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.670680    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67qr2\" (UniqueName: \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.670852 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:16.670693    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls podName:b35465e1-eb0a-45d2-92ae-6b4acca81a1a nodeName:}" failed. No retries permitted until 2026-11-02 15:19:17.17067702 +0000 UTC m=+1363.608579139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls") pod "maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" (UID: "b35465e1-eb0a-45d2-92ae-6b4acca81a1a") : secret "maas-api-e2e-ait-ad0b4b5e-serving-cert" not found
Nov 02 15:19:16.670852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.670758    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.671483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.671444    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.673485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.673453    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:16.756677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:16.756435    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67qr2\" (UniqueName: \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:17.188878 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:17.188841    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:17.191166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:17.191144    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") pod \"maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:17.428163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:17.428124    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:17.840654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:17.840617    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:19:17.841640 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:19:17.841616    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb35465e1_eb0a_45d2_92ae_6b4acca81a1a.slice/crio-12affd89c0638b7bb03595444d39dce6aff00aae2b59ca9e52d8e83a68d5f06b WatchSource:0}: Error finding container 12affd89c0638b7bb03595444d39dce6aff00aae2b59ca9e52d8e83a68d5f06b: Status 404 returned error can't find the container with id 12affd89c0638b7bb03595444d39dce6aff00aae2b59ca9e52d8e83a68d5f06b
Nov 02 15:19:18.330097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:18.330060    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" event={"ID":"b35465e1-eb0a-45d2-92ae-6b4acca81a1a","Type":"ContainerStarted","Data":"12affd89c0638b7bb03595444d39dce6aff00aae2b59ca9e52d8e83a68d5f06b"}
Nov 02 15:19:19.338396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:19.338354    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" event={"ID":"b35465e1-eb0a-45d2-92ae-6b4acca81a1a","Type":"ContainerStarted","Data":"a9ee81382030c3e10933b36c2de6809f7550a12ab7682dc651bda45707a2a07a"}
Nov 02 15:19:19.338820 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:19.338449    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:19.458670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:19.458609    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" podStartSLOduration=2.606948541 podStartE2EDuration="3.458588201s" totalImagesPullingTime="851.63966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:19:19.456579854 +0000 UTC m=+1365.894481998" watchObservedRunningTime="2026-11-02 15:19:19.458588201 +0000 UTC m=+1365.896490343"
Nov 02 15:19:21.469453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.469416    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:19:21.474632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.474606    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.564122 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.564081    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.564339 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.564239    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.564444 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.564428    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdz5k\" (UniqueName: \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.564516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.564486    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.580515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.580478    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-c82fc808-serving-cert\""
Nov 02 15:19:21.590029 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.589990    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:19:21.666025 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.665988    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.666254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.666121    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.666383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.666364    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdz5k\" (UniqueName: \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.666454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.666442    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.666602 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:21.666579    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-serving-cert: secret "maas-api-e2e-migrate-c82fc808-serving-cert" not found
Nov 02 15:19:21.666670 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:21.666654    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls podName:d4206bce-0286-4ddf-a72a-c9a7d37a68ca nodeName:}" failed. No retries permitted until 2026-11-02 15:19:22.166631439 +0000 UTC m=+1368.604533573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls") pod "maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" (UID: "d4206bce-0286-4ddf-a72a-c9a7d37a68ca") : secret "maas-api-e2e-migrate-c82fc808-serving-cert" not found
Nov 02 15:19:21.666816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.666795    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.669304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.669284    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:21.761266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:21.756096    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdz5k\" (UniqueName: \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:22.207504 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:22.207473    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:22.209941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:22.209914    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") pod \"maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:22.389703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:22.389670    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:22.771390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:22.771360    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:19:22.772469 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:19:22.772440    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4206bce_0286_4ddf_a72a_c9a7d37a68ca.slice/crio-c2d9b5db52ca3104e0af89e1e46891368643addf35dfec331cbbf04ecd2c843c WatchSource:0}: Error finding container c2d9b5db52ca3104e0af89e1e46891368643addf35dfec331cbbf04ecd2c843c: Status 404 returned error can't find the container with id c2d9b5db52ca3104e0af89e1e46891368643addf35dfec331cbbf04ecd2c843c
Nov 02 15:19:23.371991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:23.371957    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" event={"ID":"d4206bce-0286-4ddf-a72a-c9a7d37a68ca","Type":"ContainerStarted","Data":"c2d9b5db52ca3104e0af89e1e46891368643addf35dfec331cbbf04ecd2c843c"}
Nov 02 15:19:24.380246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:24.380179    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" event={"ID":"d4206bce-0286-4ddf-a72a-c9a7d37a68ca","Type":"ContainerStarted","Data":"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"}
Nov 02 15:19:24.380781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:24.380311    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:24.500371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:24.500318    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" podStartSLOduration=2.683381975 podStartE2EDuration="3.500302612s" totalImagesPullingTime="816.920637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:19:24.498586282 +0000 UTC m=+1370.936488423" watchObservedRunningTime="2026-11-02 15:19:24.500302612 +0000 UTC m=+1370.938204752"
Nov 02 15:19:25.354050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:25.354019    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:19:30.393609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:30.393576    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:19:33.659232 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:19:33.658402    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:19:53.709465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.709384    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"]
Nov 02 15:19:53.715460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.715434    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815237    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815372    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0169e1af-5286-420d-8fe8-96f88517b525-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815455    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815538    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815645    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815754    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815804    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dp66\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-kube-api-access-6dp66\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.815960 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815937    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.816001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.815981    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0169e1af-5286-420d-8fe8-96f88517b525-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.918011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.917974    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.918258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918038    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dp66\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-kube-api-access-6dp66\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.918258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918148    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.918258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918181    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0169e1af-5286-420d-8fe8-96f88517b525-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918358    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918416    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0169e1af-5286-420d-8fe8-96f88517b525-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918462    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918501    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918516    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918664    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.918842    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.919844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.919820    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0169e1af-5286-420d-8fe8-96f88517b525-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.920105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.920079    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.920410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.920356    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.923244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.922159    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"]
Nov 02 15:19:53.923244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.922423    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0169e1af-5286-420d-8fe8-96f88517b525-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.923244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.923128    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0169e1af-5286-420d-8fe8-96f88517b525-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.980497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.980401    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:53.980693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:53.980662    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dp66\" (UniqueName: \"kubernetes.io/projected/0169e1af-5286-420d-8fe8-96f88517b525-kube-api-access-6dp66\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-cw6h6\" (UID: \"0169e1af-5286-420d-8fe8-96f88517b525\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:54.030352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:54.030318    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:54.472457 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:54.472425    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"]
Nov 02 15:19:54.473074 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:19:54.473050    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0169e1af_5286_420d_8fe8_96f88517b525.slice/crio-bee6b04ec2b948ba33e78b2277d3b9fd514201a85885258e2575ed78b1918602 WatchSource:0}: Error finding container bee6b04ec2b948ba33e78b2277d3b9fd514201a85885258e2575ed78b1918602: Status 404 returned error can't find the container with id bee6b04ec2b948ba33e78b2277d3b9fd514201a85885258e2575ed78b1918602
Nov 02 15:19:54.475949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:54.475909    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:19:54.476021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:54.476000    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:19:54.476057 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:54.476031    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:19:55.003769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.003724    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" event={"ID":"0169e1af-5286-420d-8fe8-96f88517b525","Type":"ContainerStarted","Data":"0d3d763f36c13e4cbf521e81ebe3c38961b46548646363dafe3641191eaa4473"}
Nov 02 15:19:55.004323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.003790    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" event={"ID":"0169e1af-5286-420d-8fe8-96f88517b525","Type":"ContainerStarted","Data":"bee6b04ec2b948ba33e78b2277d3b9fd514201a85885258e2575ed78b1918602"}
Nov 02 15:19:55.036399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.036356    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:19:55.037864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.037814    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:19:55.038013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.037884    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:19:55.214976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:55.214917    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podStartSLOduration=2.21490162 podStartE2EDuration="2.21490162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:19:55.197729975 +0000 UTC m=+1401.635632119" watchObservedRunningTime="2026-11-02 15:19:55.21490162 +0000 UTC m=+1401.652803760"
Nov 02 15:19:56.031573 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:56.031538    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:19:56.032002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:56.031603    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:19:57.031135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:57.031100    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:19:57.031395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:57.031159    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:19:58.030736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:58.030696    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:19:58.031291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:58.030796    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:19:59.030686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:59.030650    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:19:59.030921 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:19:59.030710    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:20:00.031019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:00.030979    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:20:00.031442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:00.031038    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:20:01.030941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:01.030898    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:20:01.031136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:01.030988    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:20:02.031655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:02.031615    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused" start-of-body=
Nov 02 15:20:02.032255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:02.031692    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": dial tcp 10.133.0.95:15021: connect: connection refused"
Nov 02 15:20:03.127556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:03.127523    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:20:03.128899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:03.128871    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:20:03.129700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:03.129679    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6"
Nov 02 15:20:03.575310 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:20:03.575273    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:20:08.828250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.828211    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"]
Nov 02 15:20:08.833468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.833441    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944214    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944254    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944343    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944383    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944410    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4w2n\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-kube-api-access-g4w2n\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944715 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944687    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944821    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944863    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:08.944909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:08.944890    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.046852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.046807    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.047133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.047097    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.047345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.047326    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.047489 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.047475    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.047627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.047612    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4w2n\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-kube-api-access-g4w2n\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.047974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.047944    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.048088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.048011    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.048823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.048798    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.049376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.049354    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.049784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.049399    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.049968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.049942    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.051220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.050289    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.051220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.050356    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.051220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.050640    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.052672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.052626    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.053412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.052844    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.055327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.055300    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"]
Nov 02 15:20:09.111666 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.111562    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4w2n\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-kube-api-access-g4w2n\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.114699 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.114667    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-fxb56\" (UID: \"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.150634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.150598    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:09.593954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.593923    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"]
Nov 02 15:20:09.594887 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:20:09.594851    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf6ef44_d2ec_4e80_96bd_d3aa0cfc1396.slice/crio-bc5d57ab950290a81d4965c9ee00c617dbf7813ea80cb6c9803c8c9e05c687f0 WatchSource:0}: Error finding container bc5d57ab950290a81d4965c9ee00c617dbf7813ea80cb6c9803c8c9e05c687f0: Status 404 returned error can't find the container with id bc5d57ab950290a81d4965c9ee00c617dbf7813ea80cb6c9803c8c9e05c687f0
Nov 02 15:20:09.598293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.598113    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:09.598293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.598265    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:09.598433 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:09.598319    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:10.123337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.123300    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" event={"ID":"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396","Type":"ContainerStarted","Data":"8e1104cd19967c821904252cdf5c48599297c61d6cd642f893033bfa02b9f143"}
Nov 02 15:20:10.123746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.123364    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" event={"ID":"acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396","Type":"ContainerStarted","Data":"bc5d57ab950290a81d4965c9ee00c617dbf7813ea80cb6c9803c8c9e05c687f0"}
Nov 02 15:20:10.151818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.151777    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:10.153650 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.153625    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:10.153777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.153676    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:10.342280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:10.342227    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podStartSLOduration=2.3422107690000002 podStartE2EDuration="2.342210769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:20:10.298681694 +0000 UTC m=+1416.736583836" watchObservedRunningTime="2026-11-02 15:20:10.342210769 +0000 UTC m=+1416.780112909"
Nov 02 15:20:11.151923 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:11.151891    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:11.152336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:11.151943    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:12.151621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:12.151588    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:12.151809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:12.151654    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:13.151817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:13.151781    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:13.152362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:13.151854    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:14.150943 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:14.150908    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:14.151161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:14.150966    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:15.151313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:15.151272    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:15.151727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:15.151333    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:16.151836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:16.151799    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:16.152336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:16.151857    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:17.151280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:17.151231    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:17.151516 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:17.151293    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:18.067329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:18.067274    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"]
Nov 02 15:20:18.113134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:18.113097    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8d31fdd-5hk9g"]
Nov 02 15:20:18.151698 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:18.151656    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused" start-of-body=
Nov 02 15:20:18.151913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:18.151717    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": dial tcp 10.133.0.96:15021: connect: connection refused"
Nov 02 15:20:19.748671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:19.748627    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e070147d-3f97-41b0-a487-cdab361c9274" path="/var/lib/kubelet/pods/e070147d-3f97-41b0-a487-cdab361c9274/volumes"
Nov 02 15:20:19.995119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:19.995089    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:20.001254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:20.001163    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:20.004944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:20.004859    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56"
Nov 02 15:20:27.172369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:27.172311    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:20:27.172928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:27.172395    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:20:30.380697 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.380610    4470 generic.go:356] "Generic (PLEG): container finished" podID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerID="026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac" exitCode=0
Nov 02 15:20:30.380697 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.380683    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerDied","Data":"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"}
Nov 02 15:20:30.381249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.381232    4470 scope.go:122] "RemoveContainer" containerID="026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"
Nov 02 15:20:30.383068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.383035    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:30.383136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.383115    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:30.383172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:30.383151    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:31.403026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.402988    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerStarted","Data":"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"}
Nov 02 15:20:31.404773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.404752    4470 generic.go:356] "Generic (PLEG): container finished" podID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerID="c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881" exitCode=0
Nov 02 15:20:31.404876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.404823    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerDied","Data":"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"}
Nov 02 15:20:31.405300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.405277    4470 scope.go:122] "RemoveContainer" containerID="c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"
Nov 02 15:20:31.406926 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.406892    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:31.407003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.406982    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:31.407068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.407052    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:20:31.826994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.826538    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:20:31.826994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.826668    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:31.826994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:31.826724    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:32.608917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:32.608876    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerStarted","Data":"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"}
Nov 02 15:20:32.835692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:32.835651    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:32.836945 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:32.836911    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:33.150208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:33.150153    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:20:33.151402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:33.151372    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:33.151487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:33.151427    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:33.567367 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:20:33.567331    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:20:33.853281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:33.852836    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:33.853281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:33.852912    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:34.150077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:34.149997    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:34.150077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:34.150050    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:35.087811 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.063902    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:35.087811 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.063964    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:35.164836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.155360    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:35.164836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.155420    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:35.825735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.825688    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:35.825929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:35.825770    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:36.312871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:36.312833    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:36.334594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:36.334543    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:36.839235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:36.835591    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:36.839235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:36.835666    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:37.150149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:37.150055    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:37.150149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:37.150124    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:37.862237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:37.857281    4470 patch_prober.go:29] interesting pod/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Nov 02 15:20:37.862237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:37.857361    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Nov 02 15:20:37.960217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:37.955032    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:20:38.175028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:38.174940    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:38.192285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:38.175266    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:38.830562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:38.830530    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:20:38.831055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:38.831039    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:20:38.831730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:38.831703    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:20:39.150274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:39.150014    4470 patch_prober.go:29] interesting pod/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Nov 02 15:20:39.150274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:39.150076    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Nov 02 15:20:40.429580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:40.429543    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:20:40.430810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:20:40.430787    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:21:03.566615 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:03.566570    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:21:07.659899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:07.659864    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:21:07.660355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:07.660090    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" podUID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" containerName="maas-api" containerID="cri-o://a9ee81382030c3e10933b36c2de6809f7550a12ab7682dc651bda45707a2a07a" gracePeriod=30
Nov 02 15:21:08.101611 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.101567    4470 generic.go:356] "Generic (PLEG): container finished" podID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" containerID="a9ee81382030c3e10933b36c2de6809f7550a12ab7682dc651bda45707a2a07a" exitCode=0
Nov 02 15:21:08.102121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.102093    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" event={"ID":"b35465e1-eb0a-45d2-92ae-6b4acca81a1a","Type":"ContainerDied","Data":"a9ee81382030c3e10933b36c2de6809f7550a12ab7682dc651bda45707a2a07a"}
Nov 02 15:21:08.149839 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.149813    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:21:08.259304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.258332    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") pod \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") "
Nov 02 15:21:08.259304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.259011    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\" (UniqueName: \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\") pod \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") "
Nov 02 15:21:08.261133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.259484    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\") pod \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") "
Nov 02 15:21:08.261133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.259943    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca" pod "b35465e1-eb0a-45d2-92ae-6b4acca81a1a" (UID: "b35465e1-eb0a-45d2-92ae-6b4acca81a1a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:21:08.261133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.260226    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\") pod \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\" (UID: \"b35465e1-eb0a-45d2-92ae-6b4acca81a1a\") "
Nov 02 15:21:08.262048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.261666    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:08.278502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.278462    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2" pod "b35465e1-eb0a-45d2-92ae-6b4acca81a1a" (UID: "b35465e1-eb0a-45d2-92ae-6b4acca81a1a"). InnerVolumeSpecName "kube-api-access-67qr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:08.280385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.280349    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls" pod "b35465e1-eb0a-45d2-92ae-6b4acca81a1a" (UID: "b35465e1-eb0a-45d2-92ae-6b4acca81a1a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:21:08.295227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.292811    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs" pod "b35465e1-eb0a-45d2-92ae-6b4acca81a1a" (UID: "b35465e1-eb0a-45d2-92ae-6b4acca81a1a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:21:08.368953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.368868    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:08.369170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.369152    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67qr2\" (UniqueName: \"kubernetes.io/projected/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-kube-api-access-67qr2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:08.369310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.369296    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b35465e1-eb0a-45d2-92ae-6b4acca81a1a-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:08.955949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.955390    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:21:08.955949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:08.955665    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" podUID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" containerName="maas-api" containerID="cri-o://4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18" gracePeriod=30
Nov 02 15:21:09.114959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.114917    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t" event={"ID":"b35465e1-eb0a-45d2-92ae-6b4acca81a1a","Type":"ContainerDied","Data":"12affd89c0638b7bb03595444d39dce6aff00aae2b59ca9e52d8e83a68d5f06b"}
Nov 02 15:21:09.115137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.115001    4470 scope.go:122] "RemoveContainer" containerID="a9ee81382030c3e10933b36c2de6809f7550a12ab7682dc651bda45707a2a07a"
Nov 02 15:21:09.115278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.115263    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"
Nov 02 15:21:09.374567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.374531    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:21:09.379736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.379707    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ad0b4b5e-85d779849d-g8x5t"]
Nov 02 15:21:09.842223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.833528    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" path="/var/lib/kubelet/pods/b35465e1-eb0a-45d2-92ae-6b4acca81a1a/volumes"
Nov 02 15:21:09.964377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:09.964348    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:21:10.065397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.065367    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\") pod \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") "
Nov 02 15:21:10.065576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.065461    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\") pod \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") "
Nov 02 15:21:10.065576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.065493    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\" (UniqueName: \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\") pod \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") "
Nov 02 15:21:10.065749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.065730    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") pod \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\" (UID: \"d4206bce-0286-4ddf-a72a-c9a7d37a68ca\") "
Nov 02 15:21:10.065817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.065790    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca" pod "d4206bce-0286-4ddf-a72a-c9a7d37a68ca" (UID: "d4206bce-0286-4ddf-a72a-c9a7d37a68ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:21:10.066402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.066376    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:10.067743 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.067720    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs" pod "d4206bce-0286-4ddf-a72a-c9a7d37a68ca" (UID: "d4206bce-0286-4ddf-a72a-c9a7d37a68ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:21:10.067836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.067783    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls" pod "d4206bce-0286-4ddf-a72a-c9a7d37a68ca" (UID: "d4206bce-0286-4ddf-a72a-c9a7d37a68ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:21:10.067836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.067787    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k" pod "d4206bce-0286-4ddf-a72a-c9a7d37a68ca" (UID: "d4206bce-0286-4ddf-a72a-c9a7d37a68ca"). InnerVolumeSpecName "kube-api-access-qdz5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:10.132593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.131538    4470 generic.go:356] "Generic (PLEG): container finished" podID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" containerID="4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18" exitCode=0
Nov 02 15:21:10.132593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.131611    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"
Nov 02 15:21:10.132593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.131626    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" event={"ID":"d4206bce-0286-4ddf-a72a-c9a7d37a68ca","Type":"ContainerDied","Data":"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"}
Nov 02 15:21:10.132593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.131681    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9" event={"ID":"d4206bce-0286-4ddf-a72a-c9a7d37a68ca","Type":"ContainerDied","Data":"c2d9b5db52ca3104e0af89e1e46891368643addf35dfec331cbbf04ecd2c843c"}
Nov 02 15:21:10.132593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.131712    4470 scope.go:122] "RemoveContainer" containerID="4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"
Nov 02 15:21:10.147990 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.147972    4470 scope.go:122] "RemoveContainer" containerID="4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"
Nov 02 15:21:10.148379 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:10.148352    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18\": container with ID starting with 4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18 not found: ID does not exist" containerID="4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"
Nov 02 15:21:10.148448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.148388    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18"} err="failed to get container status \"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18\": rpc error: code = NotFound desc = could not find container \"4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18\": container with ID starting with 4cbf33c3b1318078d0b2d40dd51e97136456168d11750a1a8bcd4c4f39217a18 not found: ID does not exist"
Nov 02 15:21:10.168150 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.168116    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:10.168324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.168154    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdz5k\" (UniqueName: \"kubernetes.io/projected/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-kube-api-access-qdz5k\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:10.168324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.168169    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4206bce-0286-4ddf-a72a-c9a7d37a68ca-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:10.306222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.305893    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:21:10.340238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.331158    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-c82fc808-85f87c5fd-56kt9"]
Nov 02 15:21:10.557215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.534829    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:21:10.557215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:10.535710    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy" containerID="cri-o://a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c" gracePeriod=30
Nov 02 15:21:11.735653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.735036    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" path="/var/lib/kubelet/pods/d4206bce-0286-4ddf-a72a-c9a7d37a68ca/volumes"
Nov 02 15:21:11.762673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.762639    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:21:11.763594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763569    4470 cpu_manager.go:414] "containerMap: removing container" podUID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" containerName="maas-api"
Nov 02 15:21:11.763594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763589    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" containerName="maas-api"
Nov 02 15:21:11.763810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763631    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" containerName="maas-api"
Nov 02 15:21:11.763810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763637    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" containerName="maas-api"
Nov 02 15:21:11.763810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763751    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4206bce-0286-4ddf-a72a-c9a7d37a68ca" containerName="maas-api"
Nov 02 15:21:11.763810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.763765    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="b35465e1-eb0a-45d2-92ae-6b4acca81a1a" containerName="maas-api"
Nov 02 15:21:11.767547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.767419    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.844381 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.844347    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5cbda1cc-openshift-default-dockercfg-jk44k\""
Nov 02 15:21:11.858790 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.852913    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873523    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873625    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873669    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873746    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873822    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873850    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.874210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.873926    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwlq5\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.890225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.876929    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.890225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.876981    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.895702 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.895653    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:21:11.901964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.896247    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy" containerID="cri-o://bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6" gracePeriod=30
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981146    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981229    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981310    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981391    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981434    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981509    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981587    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981612    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.981695    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwlq5\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.982457    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.982467    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.982816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.982683    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.983632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.982939    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:11.998539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:11.983919    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:12.054271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:12.048016    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:12.076214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:12.058132    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:12.221377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:12.181745    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:12.290126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:12.249404    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwlq5\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\") pod \"e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") " pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:12.390556 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:12.390467    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:13.298214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:13.279654    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:21:13.314434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:13.307419    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:21:13.314434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:13.307533    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:21:13.314434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:13.307579    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:21:13.462935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:13.462887    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerStarted","Data":"9ab4021871f4aeee86e4eb507106876085584ed7c1fefad3f08d65af5c9bdbeb"}
Nov 02 15:21:14.507201 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:14.506022    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerStarted","Data":"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"}
Nov 02 15:21:14.721230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:14.719782    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podStartSLOduration=3.719757502 podStartE2EDuration="3.719757502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:21:14.716724735 +0000 UTC m=+1481.154626876" watchObservedRunningTime="2026-11-02 15:21:14.719757502 +0000 UTC m=+1481.157659643"
Nov 02 15:21:15.390849 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:15.390795    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:15.398643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:15.398603    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:15.514594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:15.514561    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:15.516583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:15.516554    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:21:16.145252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.144466    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:21:16.270823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.270773    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.270877    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.270947    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271001    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271336    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271408    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271447    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271499    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.271604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.271553    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\") pod \"8a7787fa-6612-4c40-b52c-20dddc76f55d\" (UID: \"8a7787fa-6612-4c40-b52c-20dddc76f55d\") "
Nov 02 15:21:16.272507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.272475    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:16.273068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.273034    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:16.273172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.273127    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:21:16.273319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.273284    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:16.273420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.273402    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:16.275097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.275058    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:16.275345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.275323    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:16.291460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.291411    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "kube-api-access-gqmk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:16.292599 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.292499    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo" pod "8a7787fa-6612-4c40-b52c-20dddc76f55d" (UID: "8a7787fa-6612-4c40-b52c-20dddc76f55d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:21:16.374357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374326    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374354    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374366    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a7787fa-6612-4c40-b52c-20dddc76f55d-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374376    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374385    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374395    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqmk7\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-kube-api-access-gqmk7\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374405    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a7787fa-6612-4c40-b52c-20dddc76f55d-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374414    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.374590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.374422    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a7787fa-6612-4c40-b52c-20dddc76f55d-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:16.522852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.522726    4470 generic.go:356] "Generic (PLEG): container finished" podID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerID="a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c" exitCode=0
Nov 02 15:21:16.522852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.522783    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"
Nov 02 15:21:16.522852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.522820    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerDied","Data":"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"}
Nov 02 15:21:16.523426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.522880    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb" event={"ID":"8a7787fa-6612-4c40-b52c-20dddc76f55d","Type":"ContainerDied","Data":"45f69be937954165b56a0b0ac2e1304308923d1f9633211c0dc3a775898fc9c4"}
Nov 02 15:21:16.523426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.522909    4470 scope.go:122] "RemoveContainer" containerID="a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"
Nov 02 15:21:16.534992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.534967    4470 scope.go:122] "RemoveContainer" containerID="c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"
Nov 02 15:21:16.545058 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.545037    4470 scope.go:122] "RemoveContainer" containerID="a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"
Nov 02 15:21:16.545425 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:16.545405    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c\": container with ID starting with a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c not found: ID does not exist" containerID="a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"
Nov 02 15:21:16.545486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.545436    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c"} err="failed to get container status \"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c\": rpc error: code = NotFound desc = could not find container \"a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c\": container with ID starting with a6e4fc054a8ef6873653a2a37bc5d277f9efc2fc5523b6351a88bb6d877d124c not found: ID does not exist"
Nov 02 15:21:16.545486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.545459    4470 scope.go:122] "RemoveContainer" containerID="c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"
Nov 02 15:21:16.545736 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:16.545709    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881\": container with ID starting with c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881 not found: ID does not exist" containerID="c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"
Nov 02 15:21:16.545780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.545742    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881"} err="failed to get container status \"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881\": rpc error: code = NotFound desc = could not find container \"c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881\": container with ID starting with c433a738b511c71475b9c2088617b72c26ebda28b5e7ec5d6e0dc0a9b89cb881 not found: ID does not exist"
Nov 02 15:21:16.694286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.694254    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:21:16.701846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:16.701815    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-ad0b4b5e-openshift-default-66df65f85f-qxmfb"]
Nov 02 15:21:17.142692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.142657    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:21:17.290383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290345    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290504    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290604    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290695 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290666    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290722    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290767    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.290898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.290850    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.291135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.291119    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.291280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.291251    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:17.291460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.291442    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\") pod \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\" (UID: \"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53\") "
Nov 02 15:21:17.291859 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.291832    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:21:17.292133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292110    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:17.292390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292367    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:17.292719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292689    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:17.292858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292832    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.292858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292860    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.293047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292874    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.293047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.292887    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.293718 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.293688    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:21:17.293972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.293947    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:21:17.294165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.294144    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:17.298986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.298953    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp" pod "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" (UID: "4d5d7e35-6e0c-4639-bad6-3a3752ba3c53"). InnerVolumeSpecName "kube-api-access-fmbbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:21:17.444213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.441814    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.444213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.441853    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.444213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.441867    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.444213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.441883    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmbbp\" (UniqueName: \"kubernetes.io/projected/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-kube-api-access-fmbbp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.444213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.441900    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:21:17.618266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.618157    4470 generic.go:356] "Generic (PLEG): container finished" podID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerID="bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6" exitCode=0
Nov 02 15:21:17.618911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.618325    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerDied","Data":"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"}
Nov 02 15:21:17.618911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.618421    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6" event={"ID":"4d5d7e35-6e0c-4639-bad6-3a3752ba3c53","Type":"ContainerDied","Data":"318db947b6628258b6fce31b963e45b6b3d1125d1d4d3ab5aad6f2594713cc40"}
Nov 02 15:21:17.618911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.618453    4470 scope.go:122] "RemoveContainer" containerID="bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"
Nov 02 15:21:17.618911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.618621    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"
Nov 02 15:21:17.757737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.748612    4470 scope.go:122] "RemoveContainer" containerID="026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"
Nov 02 15:21:17.802835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.802791    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" path="/var/lib/kubelet/pods/8a7787fa-6612-4c40-b52c-20dddc76f55d/volumes"
Nov 02 15:21:17.869427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.869004    4470 scope.go:122] "RemoveContainer" containerID="bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"
Nov 02 15:21:17.869510 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:17.869476    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6\": container with ID starting with bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6 not found: ID does not exist" containerID="bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"
Nov 02 15:21:17.869564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.869514    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6"} err="failed to get container status \"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6\": rpc error: code = NotFound desc = could not find container \"bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6\": container with ID starting with bf4212a81c9fffc5fa686671e26f11b910cb7e41dc7ca38ea2a80faee5d794b6 not found: ID does not exist"
Nov 02 15:21:17.869564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.869541    4470 scope.go:122] "RemoveContainer" containerID="026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"
Nov 02 15:21:17.874219 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:17.870073    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac\": container with ID starting with 026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac not found: ID does not exist" containerID="026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"
Nov 02 15:21:17.874219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.870110    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac"} err="failed to get container status \"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac\": rpc error: code = NotFound desc = could not find container \"026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac\": container with ID starting with 026c4489148047008179603e6bd4ec25647e7e79e8eec2e3ac6fb5344427baac not found: ID does not exist"
Nov 02 15:21:17.956215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.950245    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:21:17.958822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:17.958767    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c82fc808-gw-openshift-default-67fcd8c597-jjqh6"]
Nov 02 15:21:19.771224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:19.770652    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" path="/var/lib/kubelet/pods/4d5d7e35-6e0c-4639-bad6-3a3752ba3c53/volumes"
Nov 02 15:21:33.567417 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:21:33.567372    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:21:33.751943 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.751912    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:21:33.752803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752786    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:21:33.752866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752805    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:21:33.752866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752840    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.752866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752845    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.752866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752864    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.752994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752869    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.752994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752960    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:21:33.752994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752973    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.752994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.752982    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a7787fa-6612-4c40-b52c-20dddc76f55d" containerName="istio-proxy"
Nov 02 15:21:33.755794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.755771    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.864794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.864702    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:21:33.866394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866358    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.866533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866405    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.866533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866464    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.866684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866662    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sbwt\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.866843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866817    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.867012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.866991    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.867102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.867028    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.896947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.896919    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:21:33.906020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.905995    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:21:33.968952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.968921    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969045    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sbwt\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969120    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969274    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969309    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969399    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969529    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.969977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.969954    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.971523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.971503    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.971782 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.971764    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.971836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.971820    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.971911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.971893    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:33.974700 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:33.974678    4470 scope.go:122] "RemoveContainer" containerID="466a1659b8a0df608156befe1be51fa35e16d38a61b7c6e18b0eb6a7e2ae54a9"
Nov 02 15:21:34.030268 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.030237    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:34.030476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.030457    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sbwt\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\") pod \"istiod-openshift-gateway-7ddd9cfdc8-9d6dj\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:34.070890 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.070854    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:34.522812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.522783    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:21:34.523859 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:21:34.523831    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fae4b29_d226_4bc9_971b_bdddd302f1ca.slice/crio-8fe8bc791a75521b86642811f5229127547aa33f52df000d527e9e113dd97d48 WatchSource:0}: Error finding container 8fe8bc791a75521b86642811f5229127547aa33f52df000d527e9e113dd97d48: Status 404 returned error can't find the container with id 8fe8bc791a75521b86642811f5229127547aa33f52df000d527e9e113dd97d48
Nov 02 15:21:34.526683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.526646    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:21:34.526766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:34.526724    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:21:35.021766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:35.021728    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" event={"ID":"8fae4b29-d226-4bc9-971b-bdddd302f1ca","Type":"ContainerStarted","Data":"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"}
Nov 02 15:21:35.022277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:35.021780    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" event={"ID":"8fae4b29-d226-4bc9-971b-bdddd302f1ca","Type":"ContainerStarted","Data":"8fe8bc791a75521b86642811f5229127547aa33f52df000d527e9e113dd97d48"}
Nov 02 15:21:35.022277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:35.021879    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:35.023696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:35.023668    4470 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:21:35.023817 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:35.023720    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:21:36.031632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:36.031590    4470 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-9d6dj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:21:36.032130 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:36.031668    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:21:39.032454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:39.032417    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:21:39.259141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:21:39.259071    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" podStartSLOduration=6.259050187 podStartE2EDuration="6.259050187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:21:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:21:35.190492087 +0000 UTC m=+1501.628394230" watchObservedRunningTime="2026-11-02 15:21:39.259050187 +0000 UTC m=+1505.696952328"
Nov 02 15:22:03.568079 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:22:03.568031    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:22:21.773651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.771926    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:22:21.775381 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.775344    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:22:21.775381 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.775370    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:22:21.775579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.775504    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d5d7e35-6e0c-4639-bad6-3a3752ba3c53" containerName="istio-proxy"
Nov 02 15:22:21.779844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.779823    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.859753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.859721    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:22:21.889954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.889925    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890128 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890058    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890139    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890280    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890324    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890409    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890512 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890455    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890512 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890470    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmp84\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.890622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.890541    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.910137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.910111    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-b87e90-openshift-default-dockercfg-jb6v7\""
Nov 02 15:22:21.992584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992551    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.992753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992648    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.992753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992723    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.992880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992784    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.992880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992822    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.992880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992875    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992920    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992935    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmp84\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.992969    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.993089    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.993207    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.993315    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.993430    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.993513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.993484    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.995063 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.995041    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:21.995492 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:21.995475    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:22.056215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.056145    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:22.062495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.062469    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmp84\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\") pod \"e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") " pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:22.096969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.096950    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:22.775368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.775335    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:22:22.775881 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:22:22.775861    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc780baf4_d4aa_4f00_b000_589f3b6a197f.slice/crio-8a5dedd302a5f51e7993b3c79a6d8b1929cfa231516e1b5e8061ebe9d0b2808b WatchSource:0}: Error finding container 8a5dedd302a5f51e7993b3c79a6d8b1929cfa231516e1b5e8061ebe9d0b2808b: Status 404 returned error can't find the container with id 8a5dedd302a5f51e7993b3c79a6d8b1929cfa231516e1b5e8061ebe9d0b2808b
Nov 02 15:22:22.778688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.778651    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:22:22.778784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.778724    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:22:22.778784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:22.778753    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:22:23.319687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:23.319647    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerStarted","Data":"aa2c3012d48e234f7d157fc0b042c19f5bdd9654a4c602d9b748c366ce644d22"}
Nov 02 15:22:23.319868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:23.319720    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerStarted","Data":"8a5dedd302a5f51e7993b3c79a6d8b1929cfa231516e1b5e8061ebe9d0b2808b"}
Nov 02 15:22:23.444204 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:23.444144    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podStartSLOduration=2.444126432 podStartE2EDuration="2.444126432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:22:23.440446012 +0000 UTC m=+1549.878348156" watchObservedRunningTime="2026-11-02 15:22:23.444126432 +0000 UTC m=+1549.882028572"
Nov 02 15:22:24.097814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:24.097777    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:24.103942 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:24.103914    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:24.326404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:24.326363    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:24.327547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:24.327526    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:22:33.568160 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:22:33.568099    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:22:37.064937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:37.064905    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:22:37.065379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:37.065117    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy" containerID="cri-o://3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957" gracePeriod=30
Nov 02 15:22:37.087314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:37.087282    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:22:37.087509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:37.087486    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy" containerID="cri-o://0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2" gracePeriod=30
Nov 02 15:22:42.417769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.417739    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:22:42.456306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.456277    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:22:42.465630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465603    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465646    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465729    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465775    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465836    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465919    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.465999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.465976    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.466111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466010    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.466111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466052    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.466111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466079    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.466288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466133    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.466288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466218    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.466288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466226    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\") pod \"7022fdf6-b3a9-41c2-b03d-7dc044256591\" (UID: \"7022fdf6-b3a9-41c2-b03d-7dc044256591\") "
Nov 02 15:22:42.466485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.466458    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:22:42.467210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.467172    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.467342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.467241    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.467486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.467468    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.467640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.467490    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.467640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.467507    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7022fdf6-b3a9-41c2-b03d-7dc044256591-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.468177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.468155    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:22:42.468306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.468230    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:22:42.468638 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.468612    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "kube-api-access-xlkb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:22:42.468722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.468640    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy" pod "7022fdf6-b3a9-41c2-b03d-7dc044256591" (UID: "7022fdf6-b3a9-41c2-b03d-7dc044256591"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.568240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568208    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568254    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568276    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568309    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568503    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568563    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568600    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.568696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568613    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.568696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568679    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.568822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568750    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.568889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568866    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:22:42.568929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568872    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.569015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.568998    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\") pod \"349f2413-a6d8-483a-be3b-4b496709914c\" (UID: \"349f2413-a6d8-483a-be3b-4b496709914c\") "
Nov 02 15:22:42.569343 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569314    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.569939 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569918    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569944    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569961    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569975    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.569988    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570002    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570015    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349f2413-a6d8-483a-be3b-4b496709914c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570030    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlkb6\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-kube-api-access-xlkb6\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570043    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7022fdf6-b3a9-41c2-b03d-7dc044256591-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.570681 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570658    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "kube-api-access-nhkh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:22:42.570873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.570855    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:22:42.571075 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.571049    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:22:42.571169 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.571063    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy" pod "349f2413-a6d8-483a-be3b-4b496709914c" (UID: "349f2413-a6d8-483a-be3b-4b496709914c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:22:42.645805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.645710    4470 generic.go:356] "Generic (PLEG): container finished" podID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerID="3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957" exitCode=0
Nov 02 15:22:42.645974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.645809    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"
Nov 02 15:22:42.645974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.645829    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" event={"ID":"7022fdf6-b3a9-41c2-b03d-7dc044256591","Type":"ContainerDied","Data":"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"}
Nov 02 15:22:42.645974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.645894    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7" event={"ID":"7022fdf6-b3a9-41c2-b03d-7dc044256591","Type":"ContainerDied","Data":"29f635222db3a006d7eceda2a9743c2e96627b615ac75bf9e3c81400a1d6c01f"}
Nov 02 15:22:42.645974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.645926    4470 scope.go:122] "RemoveContainer" containerID="3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"
Nov 02 15:22:42.647950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.647925    4470 generic.go:356] "Generic (PLEG): container finished" podID="349f2413-a6d8-483a-be3b-4b496709914c" containerID="0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2" exitCode=0
Nov 02 15:22:42.648088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.647987    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"
Nov 02 15:22:42.648152 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.648103    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" event={"ID":"349f2413-a6d8-483a-be3b-4b496709914c","Type":"ContainerDied","Data":"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"}
Nov 02 15:22:42.648251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.648199    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw" event={"ID":"349f2413-a6d8-483a-be3b-4b496709914c","Type":"ContainerDied","Data":"244fee10ab606c3e746d2cfdabda23c005293422a2a71447cae035e3dd35c52d"}
Nov 02 15:22:42.659468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.659390    4470 scope.go:122] "RemoveContainer" containerID="3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"
Nov 02 15:22:42.660357 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:22:42.659882    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957\": container with ID starting with 3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957 not found: ID does not exist" containerID="3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"
Nov 02 15:22:42.660357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.659916    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957"} err="failed to get container status \"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957\": rpc error: code = NotFound desc = could not find container \"3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957\": container with ID starting with 3d250371a817413da9c6a697a8c116088d346534573c8c719b444f0c25fef957 not found: ID does not exist"
Nov 02 15:22:42.660357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.659941    4470 scope.go:122] "RemoveContainer" containerID="0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"
Nov 02 15:22:42.671060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671032    4470 scope.go:122] "RemoveContainer" containerID="0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"
Nov 02 15:22:42.671477 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:22:42.671449    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2\": container with ID starting with 0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2 not found: ID does not exist" containerID="0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"
Nov 02 15:22:42.671571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671487    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2"} err="failed to get container status \"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2\": rpc error: code = NotFound desc = could not find container \"0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2\": container with ID starting with 0d2592d265eda624749bd0a051c711e0d68a9c85b2942bf1f77522a6581478d2 not found: ID does not exist"
Nov 02 15:22:42.671692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671676    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhkh5\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-kube-api-access-nhkh5\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.671745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671694    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349f2413-a6d8-483a-be3b-4b496709914c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.671745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671704    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349f2413-a6d8-483a-be3b-4b496709914c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.671745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.671713    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349f2413-a6d8-483a-be3b-4b496709914c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:22:42.860265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.859937    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:22:42.865438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.865410    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-wxsdw"]
Nov 02 15:22:42.976257 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.976209    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:22:42.980732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:42.980698    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-9tlm7"]
Nov 02 15:22:43.708162 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:43.708129    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="349f2413-a6d8-483a-be3b-4b496709914c" path="/var/lib/kubelet/pods/349f2413-a6d8-483a-be3b-4b496709914c/volumes"
Nov 02 15:22:43.708727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:22:43.708711    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" path="/var/lib/kubelet/pods/7022fdf6-b3a9-41c2-b03d-7dc044256591/volumes"
Nov 02 15:23:01.176400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.175474    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": read tcp 10.133.0.2:43786->10.133.0.97:15021: read: connection reset by peer" start-of-body=
Nov 02 15:23:01.176400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.175639    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": read tcp 10.133.0.2:43786->10.133.0.97:15021: read: connection reset by peer"
Nov 02 15:23:01.909665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.909630    4470 generic.go:356] "Generic (PLEG): container finished" podID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerID="f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0" exitCode=0
Nov 02 15:23:01.909855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.909706    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerDied","Data":"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"}
Nov 02 15:23:01.910214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.910175    4470 scope.go:122] "RemoveContainer" containerID="f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"
Nov 02 15:23:01.912029 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.912001    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:23:01.912092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.912069    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:23:01.912130 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:01.912101    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:23:02.301646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.301611    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:23:02.302398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302379    4470 cpu_manager.go:414] "containerMap: removing container" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy"
Nov 02 15:23:02.302468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302399    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy"
Nov 02 15:23:02.302468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302436    4470 cpu_manager.go:414] "containerMap: removing container" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy"
Nov 02 15:23:02.302468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302441    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy"
Nov 02 15:23:02.302604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302564    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="7022fdf6-b3a9-41c2-b03d-7dc044256591" containerName="istio-proxy"
Nov 02 15:23:02.302604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.302581    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="349f2413-a6d8-483a-be3b-4b496709914c" containerName="istio-proxy"
Nov 02 15:23:02.305371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.305353    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.349925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.349887    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.350428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.350110    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.350428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.350252    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.350428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.350320    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jngf\" (UniqueName: \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.361357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.361324    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-be5d503e-serving-cert\""
Nov 02 15:23:02.371279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.371244    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:23:02.452604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.452567    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.452986 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:02.452890    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-serving-cert: secret "maas-api-e2e-mt-be5d503e-serving-cert" not found
Nov 02 15:23:02.453361 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:02.453342    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls podName:a47637d2-49bf-4fe8-a5f3-7d19d734c8b1 nodeName:}" failed. No retries permitted until 2026-11-02 15:23:02.95330114 +0000 UTC m=+1589.391203284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls") pod "maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" (UID: "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1") : secret "maas-api-e2e-mt-be5d503e-serving-cert" not found
Nov 02 15:23:02.453519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.453075    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.453758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.453737    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.453913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.453896    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jngf\" (UniqueName: \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.455312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.455286    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.456295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.456243    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.562269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.554892    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jngf\" (UniqueName: \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.944067 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.940178    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerStarted","Data":"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"}
Nov 02 15:23:02.992211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.976821    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:02.999178 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:02.999135    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") pod \"maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:03.244664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.244580    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:03.400612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.398441    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:23:03.400612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.399041    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:03.400612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.399126    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:03.567672 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:03.567579    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:23:03.662342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.662304    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:23:03.662609 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:23:03.662573    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47637d2_49bf_4fe8_a5f3_7d19d734c8b1.slice/crio-4eee62c10a9c7bc80679f3c22e8335289e632c5d6af3a6b9c919ca2f889c7c45 WatchSource:0}: Error finding container 4eee62c10a9c7bc80679f3c22e8335289e632c5d6af3a6b9c919ca2f889c7c45: Status 404 returned error can't find the container with id 4eee62c10a9c7bc80679f3c22e8335289e632c5d6af3a6b9c919ca2f889c7c45
Nov 02 15:23:03.664680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.664657    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:23:03.959664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:03.959626    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" event={"ID":"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1","Type":"ContainerStarted","Data":"4eee62c10a9c7bc80679f3c22e8335289e632c5d6af3a6b9c919ca2f889c7c45"}
Nov 02 15:23:04.393280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:04.391119    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:04.393280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:04.391214    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:04.995242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:04.989536    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" event={"ID":"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1","Type":"ContainerStarted","Data":"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"}
Nov 02 15:23:04.995242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:04.990864    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:05.111113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:05.111062    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" podStartSLOduration=2.246463376 podStartE2EDuration="3.111047286s" totalImagesPullingTime="864.58391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:23:05.108314692 +0000 UTC m=+1591.546216836" watchObservedRunningTime="2026-11-02 15:23:05.111047286 +0000 UTC m=+1591.548949427"
Nov 02 15:23:05.400037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:05.399938    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:05.400410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:05.400367    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:06.391886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:06.391846    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:06.392567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:06.392529    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:07.393646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:07.393606    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:07.394286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:07.394250    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:08.399855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:08.399801    4470 patch_prober.go:29] interesting pod/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused" start-of-body=
Nov 02 15:23:08.400557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:08.400522    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.97:15021/healthz/ready\": dial tcp 10.133.0.97:15021: connect: connection refused"
Nov 02 15:23:09.395622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:09.395591    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:23:09.395899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:09.395877    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:23:09.396769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:09.396749    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:23:12.027126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:12.027098    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:23:13.354333 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.354300    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:23:13.357706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.357682    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.427866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.427825    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-895ac8-serving-cert\""
Nov 02 15:23:13.448382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.448330    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:23:13.479682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.478778    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.479682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.479000    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b6dr\" (UniqueName: \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.479682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.479064    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.479682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.479456    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.593004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.586480    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.593004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.586682    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b6dr\" (UniqueName: \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.593004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.586762    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.593004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.586935    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.599261 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:13.599222    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-serving-cert: secret "maas-api-e2e-shared-a-w1-895ac8-serving-cert" not found
Nov 02 15:23:13.599472 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:13.599323    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls podName:79bdf7f0-47cb-446e-aa6a-4faea8a71a49 nodeName:}" failed. No retries permitted until 2026-11-02 15:23:14.099299468 +0000 UTC m=+1600.537201599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls") pod "maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" (UID: "79bdf7f0-47cb-446e-aa6a-4faea8a71a49") : secret "maas-api-e2e-shared-a-w1-895ac8-serving-cert" not found
Nov 02 15:23:13.600337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.600308    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.608877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.608792    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:13.798589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:13.798551    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b6dr\" (UniqueName: \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:14.116497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:14.116459    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:14.119020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:14.118993    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:14.271694 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:14.271638    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:14.767576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:14.760645    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:23:15.191548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:15.191229    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" event={"ID":"79bdf7f0-47cb-446e-aa6a-4faea8a71a49","Type":"ContainerStarted","Data":"860be9ed1bd9fdc651f8864931fe95e2a4cba60520ccbcbad3fa2ebe2b432d90"}
Nov 02 15:23:16.200357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:16.200318    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" event={"ID":"79bdf7f0-47cb-446e-aa6a-4faea8a71a49","Type":"ContainerStarted","Data":"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"}
Nov 02 15:23:16.200780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:16.200387    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:16.324689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:16.324627    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" podStartSLOduration=2.437867358 podStartE2EDuration="3.324610244s" totalImagesPullingTime="886.742886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:23:16.319705206 +0000 UTC m=+1602.757607356" watchObservedRunningTime="2026-11-02 15:23:16.324610244 +0000 UTC m=+1602.762512385"
Nov 02 15:23:22.213563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:23:22.213529    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:23:33.569494 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:23:33.569451    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:24:02.959767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:02.959727    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:24:02.960167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:02.960009    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy" containerID="cri-o://7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c" gracePeriod=30
Nov 02 15:24:03.746965 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:03.746927    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:24:08.298469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.298440    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:24:08.316111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.316075    4470 generic.go:356] "Generic (PLEG): container finished" podID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerID="7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c" exitCode=0
Nov 02 15:24:08.316328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.316136    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"
Nov 02 15:24:08.316328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.316155    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerDied","Data":"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"}
Nov 02 15:24:08.316328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.316232    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2" event={"ID":"4b9354a4-aa6d-488a-b9e2-2535f6a82db6","Type":"ContainerDied","Data":"9ab4021871f4aeee86e4eb507106876085584ed7c1fefad3f08d65af5c9bdbeb"}
Nov 02 15:24:08.316328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.316258    4470 scope.go:122] "RemoveContainer" containerID="7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"
Nov 02 15:24:08.328313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.328288    4470 scope.go:122] "RemoveContainer" containerID="f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"
Nov 02 15:24:08.338804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.338781    4470 scope.go:122] "RemoveContainer" containerID="7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"
Nov 02 15:24:08.339178 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:08.339156    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c\": container with ID starting with 7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c not found: ID does not exist" containerID="7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"
Nov 02 15:24:08.339315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.339234    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c"} err="failed to get container status \"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c\": rpc error: code = NotFound desc = could not find container \"7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c\": container with ID starting with 7786f59c370d57832cb7da7c3f0418e89150c655eb3a090f6aa8610834b8e57c not found: ID does not exist"
Nov 02 15:24:08.339315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.339262    4470 scope.go:122] "RemoveContainer" containerID="f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"
Nov 02 15:24:08.339586 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:08.339560    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0\": container with ID starting with f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0 not found: ID does not exist" containerID="f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"
Nov 02 15:24:08.339660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.339599    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0"} err="failed to get container status \"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0\": rpc error: code = NotFound desc = could not find container \"f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0\": container with ID starting with f2f43f97676908e28ea70d6fbba73b4dbc1f04c683416fde55c303170ec9fdf0 not found: ID does not exist"
Nov 02 15:24:08.416009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.415971    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416022    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416137    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416387 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416369    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416402    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416452    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416491    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416520    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:08.416609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416432    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:08.416728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416629    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416669    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:08.416835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416740    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\") pod \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\" (UID: \"4b9354a4-aa6d-488a-b9e2-2535f6a82db6\") "
Nov 02 15:24:08.416835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416751    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:08.416835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.416816    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:08.417463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.417428    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.417463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.417453    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.417463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.417463    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.417725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.417472    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.417725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.417480    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.418759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.418712    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:08.442843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.442793    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "kube-api-access-zwlq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:08.443243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.443153    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:08.443377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.443314    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo" pod "4b9354a4-aa6d-488a-b9e2-2535f6a82db6" (UID: "4b9354a4-aa6d-488a-b9e2-2535f6a82db6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:08.519411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.519323    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.519411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.519356    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwlq5\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-kube-api-access-zwlq5\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.519411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.519366    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.519411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.519375    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9354a4-aa6d-488a-b9e2-2535f6a82db6-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:08.748650 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.748613    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:24:08.754762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:08.754731    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5cbda1cc-openshift-default-5c4f9d6b9c-bx8v2"]
Nov 02 15:24:09.708579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:09.708542    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" path="/var/lib/kubelet/pods/4b9354a4-aa6d-488a-b9e2-2535f6a82db6/volumes"
Nov 02 15:24:22.853519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.853431    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:24:22.903068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.903022    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:24:22.903354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.903308    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" containerID="cri-o://83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c" gracePeriod=30
Nov 02 15:24:22.907648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.907625    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:24:22.907863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.907834    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy" containerID="cri-o://a9e84e41e9e3b11836cfbcd059a2ff50af58b4de6c6d369f2cd6112490ed7a7a" gracePeriod=30
Nov 02 15:24:22.953618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.953568    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:24:22.953921 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.953873    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" containerID="cri-o://1e81b4a0a95fd3d21164e1ba94e40bd39576be5c3870587822ae2a26f09ff662" gracePeriod=30
Nov 02 15:24:22.958256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.958171    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:24:22.958524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.958494    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy" containerID="cri-o://c35baba076ca88fcf4a32a2eb96269366d9cbc3934e0cf22624f4568679bf078" gracePeriod=30
Nov 02 15:24:22.965151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.965118    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:24:22.965518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.965435    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy" containerID="cri-o://5232b0e381fca051d9eb7a9f4219d24fd50633c875f953fc0c4127ddccf0fbab" gracePeriod=30
Nov 02 15:24:22.971139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.971116    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:24:22.971343 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:22.971322    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" containerID="cri-o://b05236150165be48ae5cfb5b99479f708facd6e634e911aa5dd314b3df0f65ca" gracePeriod=30
Nov 02 15:24:23.028647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.028599    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:24:23.028982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.028920    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy" containerID="cri-o://b58fe1f21700dbe6c5cea687cda1b749e4cd49d8d7f8afcdc22e98947050b219" gracePeriod=30
Nov 02 15:24:23.033949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.033914    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:24:23.036348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.034480    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" containerID="cri-o://d069bf55a2604e9f00f408f56096080c4a56f59a5adde685dda533bc39865147" gracePeriod=30
Nov 02 15:24:23.040287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.039477    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:24:23.040287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.039705    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy" containerID="cri-o://a97ab39269b2a1ea61a8bee2a9a6ffa01f30cc7d8d5c86f058e4cc22065b8873" gracePeriod=30
Nov 02 15:24:23.044623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.044597    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:24:23.044944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.044791    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy" containerID="cri-o://bedfc0f48d6bcc7cc5dd6640a87ebe0c8e0051a3f31bd517bda2679e1268fde4" gracePeriod=30
Nov 02 15:24:23.050136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.049812    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:24:23.050136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.049994    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy" containerID="cri-o://af23046874219a126da818ddb3143eba7c7ab4bccc9435c4770af511f886b989" gracePeriod=30
Nov 02 15:24:23.055122 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.055079    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:24:23.055483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.055454    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" containerID="cri-o://190efd00b2fff32c1cf328bd363fb6e0b0aca6baf6912941c5c262124d2bbf90" gracePeriod=30
Nov 02 15:24:23.060305 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.060277    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:24:23.060532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.060489    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy" containerID="cri-o://22ae92078e07bfc3def4e6d735d80b56f8ce6ba37e216e3a63851fa1dd42ed96" gracePeriod=30
Nov 02 15:24:23.064608 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.064584    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:24:23.065205 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.065146    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" containerID="cri-o://0b2e054a6e35f55d8758645cdd3e6b1a5b3adc243070918e7be800e72e067ff9" gracePeriod=30
Nov 02 15:24:23.068386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.068085    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:24:23.068386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.068265    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy" containerID="cri-o://16ab0570d68f48dfc2508d83e562ee0d55f26fca8cfbdd0a2e621a568447568d" gracePeriod=30
Nov 02 15:24:23.643096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:23.643047    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" containerID="cri-o://5496d447f6ae23f9d899b11a7081ff9c5064d3743698d94cf1ac2034f3d9869c" gracePeriod=30
Nov 02 15:24:25.893404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.888526    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:24:25.894217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.894166    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" containerID="cri-o://651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c" gracePeriod=30
Nov 02 15:24:25.895245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.895160    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:24:25.895523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.895459    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" containerID="cri-o://68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3" gracePeriod=30
Nov 02 15:24:25.912078 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.912042    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:24:25.917062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.917021    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:24:25.917515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.917472    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy" containerID="cri-o://1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323" gracePeriod=30
Nov 02 15:24:25.917940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:25.917912    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy" containerID="cri-o://a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e" gracePeriod=30
Nov 02 15:24:26.498570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.498240    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:24:26.499484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.499453    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.499697 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.499677    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.499910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.499895    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.500013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.500001    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.500413 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.500391    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.501486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.501462    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b9354a4-aa6d-488a-b9e2-2535f6a82db6" containerName="istio-proxy"
Nov 02 15:24:26.505142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.505114    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:26.505342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.505314    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:26.534452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.534345    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:24:26.534651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.534496    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.586291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.586182    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.586632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.586606    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.586906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.586881    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587096    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587239    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587335    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-fd1339-openshift-default-dockercfg-l99zr\""
Nov 02 15:24:26.587374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587338    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587491 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587457    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587528    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.587621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.587600    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ssw\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.639290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.639251    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:26.639459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.639328    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:26.688811 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.688772    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.688987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.688833    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.688987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.688858    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.688987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.688899    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.688987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.688951    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ssw\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689206 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689042    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689208    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689282    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689310    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689394    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689391    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689462    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.689612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.689595    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.690062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.690038    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.691476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.691454    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.691685 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.691665    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.879297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.872966    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ssw\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.935046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.935006    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\") pod \"e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") " pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:26.935857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.935831    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:26.936051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.936026    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:26.987483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.987442    4470 patch_prober.go:29] interesting pod/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:26.987762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:26.987732    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:27.203762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.203625    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:27.420366 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.420306    4470 patch_prober.go:29] interesting pod/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:27.420635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.420602    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:27.891661 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.891571    4470 patch_prober.go:29] interesting pod/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:27.891832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.891650    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:27.954499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:27.954468    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:24:28.090996 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.090952    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:28.091149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.091053    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:28.091149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.091105    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:28.701939 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.701433    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:24:28.872221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872132    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872284    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872464 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872452    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872536 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872511    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872552    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872582    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872666    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872696    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.872807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.872734    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\") pod \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\" (UID: \"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3\") "
Nov 02 15:24:28.873039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.873014    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:28.873969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.873255    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:28.873969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.873502    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:28.873969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.873698    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.875967    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.875993    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.876007    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.876020    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.876534    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.876634    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:28.884149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.876856    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:28.890524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.890464    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "kube-api-access-wcfdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:28.892235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.891114    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy" pod "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" (UID: "4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.924770    4470 generic.go:356] "Generic (PLEG): container finished" podID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerID="83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c" exitCode=0
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.924824    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.924905    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" event={"ID":"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3","Type":"ContainerDied","Data":"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.924956    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75" event={"ID":"4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3","Type":"ContainerDied","Data":"62d86380dfb1a20941f227652470a5c08ee2b2d8d40eb89e22bee3465d589ad6"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.924981    4470 scope.go:122] "RemoveContainer" containerID="83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.928513    4470 generic.go:356] "Generic (PLEG): container finished" podID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerID="5496d447f6ae23f9d899b11a7081ff9c5064d3743698d94cf1ac2034f3d9869c" exitCode=0
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.928613    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" event={"ID":"f348580c-2b74-46bf-8890-e86cfbbc0c19","Type":"ContainerDied","Data":"5496d447f6ae23f9d899b11a7081ff9c5064d3743698d94cf1ac2034f3d9869c"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.930538    4470 generic.go:356] "Generic (PLEG): container finished" podID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerID="1e81b4a0a95fd3d21164e1ba94e40bd39576be5c3870587822ae2a26f09ff662" exitCode=0
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.930617    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" event={"ID":"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4","Type":"ContainerDied","Data":"1e81b4a0a95fd3d21164e1ba94e40bd39576be5c3870587822ae2a26f09ff662"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.930721    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp" event={"ID":"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4","Type":"ContainerDied","Data":"dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.930751    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfccd694e518b686d125302636124f4ab1a07ce585a0e8ce8c1fe905f8ff88b0"
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.932547    4470 generic.go:356] "Generic (PLEG): container finished" podID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerID="a9e84e41e9e3b11836cfbcd059a2ff50af58b4de6c6d369f2cd6112490ed7a7a" exitCode=0
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.932636    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" event={"ID":"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17","Type":"ContainerDied","Data":"a9e84e41e9e3b11836cfbcd059a2ff50af58b4de6c6d369f2cd6112490ed7a7a"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.932673    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j" event={"ID":"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17","Type":"ContainerDied","Data":"d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5"}
Nov 02 15:24:28.932937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.932698    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7e86b28f4e3363db44c154a5160b1e8f835eb1de07fce2f3a448f8ae7e76dd5"
Nov 02 15:24:28.934518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.934491    4470 generic.go:356] "Generic (PLEG): container finished" podID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerID="a97ab39269b2a1ea61a8bee2a9a6ffa01f30cc7d8d5c86f058e4cc22065b8873" exitCode=0
Nov 02 15:24:28.934925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.934559    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" event={"ID":"4aa1e379-8d3b-4359-8fba-257fd79483c4","Type":"ContainerDied","Data":"a97ab39269b2a1ea61a8bee2a9a6ffa01f30cc7d8d5c86f058e4cc22065b8873"}
Nov 02 15:24:28.934925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.934616    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527" event={"ID":"4aa1e379-8d3b-4359-8fba-257fd79483c4","Type":"ContainerDied","Data":"ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b"}
Nov 02 15:24:28.934925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.934640    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ffd77252751af3d8f9b24a8f4b980329953d72c422c3e039e8e8d69ebf50404b"
Nov 02 15:24:28.936645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.936624    4470 generic.go:356] "Generic (PLEG): container finished" podID="81c60443-8c58-466a-ae68-95b76e45de67" containerID="190efd00b2fff32c1cf328bd363fb6e0b0aca6baf6912941c5c262124d2bbf90" exitCode=0
Nov 02 15:24:28.936728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.936695    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" event={"ID":"81c60443-8c58-466a-ae68-95b76e45de67","Type":"ContainerDied","Data":"190efd00b2fff32c1cf328bd363fb6e0b0aca6baf6912941c5c262124d2bbf90"}
Nov 02 15:24:28.936728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.936724    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f" event={"ID":"81c60443-8c58-466a-ae68-95b76e45de67","Type":"ContainerDied","Data":"eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8"}
Nov 02 15:24:28.937002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.936739    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eeb987f179d48b0f0cc4c052491461468cf749842b9c38adcf41c85a6e6a14f8"
Nov 02 15:24:28.938561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.938542    4470 generic.go:356] "Generic (PLEG): container finished" podID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerID="b05236150165be48ae5cfb5b99479f708facd6e634e911aa5dd314b3df0f65ca" exitCode=0
Nov 02 15:24:28.938643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.938612    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" event={"ID":"4641d0dc-3dca-4066-8222-3d9b59dd2658","Type":"ContainerDied","Data":"b05236150165be48ae5cfb5b99479f708facd6e634e911aa5dd314b3df0f65ca"}
Nov 02 15:24:28.938706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.938655    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6" event={"ID":"4641d0dc-3dca-4066-8222-3d9b59dd2658","Type":"ContainerDied","Data":"887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a"}
Nov 02 15:24:28.938706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.938678    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="887593e9c07f4d1f58afdcdca0fa5a1f510507046e4f21bd495a3dba82d74d3a"
Nov 02 15:24:28.939935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.939914    4470 generic.go:356] "Generic (PLEG): container finished" podID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerID="d069bf55a2604e9f00f408f56096080c4a56f59a5adde685dda533bc39865147" exitCode=0
Nov 02 15:24:28.940020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.939983    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" event={"ID":"62ea1a00-4254-44b4-9e3d-ca1a322e89de","Type":"ContainerDied","Data":"d069bf55a2604e9f00f408f56096080c4a56f59a5adde685dda533bc39865147"}
Nov 02 15:24:28.940570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.940039    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" event={"ID":"62ea1a00-4254-44b4-9e3d-ca1a322e89de","Type":"ContainerDied","Data":"8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5"}
Nov 02 15:24:28.940570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.940056    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8271d1fc9161900cc678584e228af7a3568a9ee7265557f0fb0c12f037e485b5"
Nov 02 15:24:28.941880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.941773    4470 generic.go:356] "Generic (PLEG): container finished" podID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerID="16ab0570d68f48dfc2508d83e562ee0d55f26fca8cfbdd0a2e621a568447568d" exitCode=0
Nov 02 15:24:28.941880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.941832    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" event={"ID":"e3ed6bbc-337d-431b-bc04-1736db28c879","Type":"ContainerDied","Data":"16ab0570d68f48dfc2508d83e562ee0d55f26fca8cfbdd0a2e621a568447568d"}
Nov 02 15:24:28.942097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.941878    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv" event={"ID":"e3ed6bbc-337d-431b-bc04-1736db28c879","Type":"ContainerDied","Data":"aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1"}
Nov 02 15:24:28.942097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.941920    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa0f36a7251f682be20441cd7738429477639e01795cc44501032e247dde2ca1"
Nov 02 15:24:28.943787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.943764    4470 generic.go:356] "Generic (PLEG): container finished" podID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerID="af23046874219a126da818ddb3143eba7c7ab4bccc9435c4770af511f886b989" exitCode=0
Nov 02 15:24:28.943896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.943843    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerDied","Data":"af23046874219a126da818ddb3143eba7c7ab4bccc9435c4770af511f886b989"}
Nov 02 15:24:28.943896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.943883    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6" event={"ID":"d31c3a95-47cd-4135-9009-2e8fc2320881","Type":"ContainerDied","Data":"394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628"}
Nov 02 15:24:28.944095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.943906    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="394fa5a6f999f4721dbf471dc669d3a405ce6473366a2daca9412e97958e2628"
Nov 02 15:24:28.945527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.945506    4470 generic.go:356] "Generic (PLEG): container finished" podID="45f12615-7aa9-4452-81eb-d7122a681047" containerID="0b2e054a6e35f55d8758645cdd3e6b1a5b3adc243070918e7be800e72e067ff9" exitCode=0
Nov 02 15:24:28.945623 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.945593    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" event={"ID":"45f12615-7aa9-4452-81eb-d7122a681047","Type":"ContainerDied","Data":"0b2e054a6e35f55d8758645cdd3e6b1a5b3adc243070918e7be800e72e067ff9"}
Nov 02 15:24:28.947750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.945632    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft" event={"ID":"45f12615-7aa9-4452-81eb-d7122a681047","Type":"ContainerDied","Data":"88833f3b9ba5e10e8f6638cd1083aa6409dc09643a38ee63ea6b484b5682f032"}
Nov 02 15:24:28.947750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.945656    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88833f3b9ba5e10e8f6638cd1083aa6409dc09643a38ee63ea6b484b5682f032"
Nov 02 15:24:28.949530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.948835    4470 generic.go:356] "Generic (PLEG): container finished" podID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerID="c35baba076ca88fcf4a32a2eb96269366d9cbc3934e0cf22624f4568679bf078" exitCode=0
Nov 02 15:24:28.949530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.948918    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" event={"ID":"10572e2d-8698-4f35-8c23-8437bbd65edb","Type":"ContainerDied","Data":"c35baba076ca88fcf4a32a2eb96269366d9cbc3934e0cf22624f4568679bf078"}
Nov 02 15:24:28.949530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.948969    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh" event={"ID":"10572e2d-8698-4f35-8c23-8437bbd65edb","Type":"ContainerDied","Data":"2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780"}
Nov 02 15:24:28.949530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.948994    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c9781b72d9225e805fceec851e95628e5f2e90c74209bc93299f53b6a37e780"
Nov 02 15:24:28.953847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.951801    4470 generic.go:356] "Generic (PLEG): container finished" podID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerID="b58fe1f21700dbe6c5cea687cda1b749e4cd49d8d7f8afcdc22e98947050b219" exitCode=0
Nov 02 15:24:28.953847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.951874    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" event={"ID":"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784","Type":"ContainerDied","Data":"b58fe1f21700dbe6c5cea687cda1b749e4cd49d8d7f8afcdc22e98947050b219"}
Nov 02 15:24:28.953847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.951909    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4" event={"ID":"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784","Type":"ContainerDied","Data":"929891014ff173247ed78682e3a24fb7c78d5105c3bcf0069f7d730409f896ca"}
Nov 02 15:24:28.953847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.951931    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="929891014ff173247ed78682e3a24fb7c78d5105c3bcf0069f7d730409f896ca"
Nov 02 15:24:28.954048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.954018    4470 generic.go:356] "Generic (PLEG): container finished" podID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerID="bedfc0f48d6bcc7cc5dd6640a87ebe0c8e0051a3f31bd517bda2679e1268fde4" exitCode=0
Nov 02 15:24:28.954110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.954089    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" event={"ID":"9ad75510-0fdf-4413-9225-87e9b3d6bd54","Type":"ContainerDied","Data":"bedfc0f48d6bcc7cc5dd6640a87ebe0c8e0051a3f31bd517bda2679e1268fde4"}
Nov 02 15:24:28.954151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.954130    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7" event={"ID":"9ad75510-0fdf-4413-9225-87e9b3d6bd54","Type":"ContainerDied","Data":"19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27"}
Nov 02 15:24:28.954216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.954154    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19a59a7e1561f96b8bd653b8476e197a23dfcd10e67b151d5606c1615ea55e27"
Nov 02 15:24:28.956352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.955912    4470 generic.go:356] "Generic (PLEG): container finished" podID="15733bb9-1506-4ab3-b622-217c34be5091" containerID="22ae92078e07bfc3def4e6d735d80b56f8ce6ba37e216e3a63851fa1dd42ed96" exitCode=0
Nov 02 15:24:28.956352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.955998    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerDied","Data":"22ae92078e07bfc3def4e6d735d80b56f8ce6ba37e216e3a63851fa1dd42ed96"}
Nov 02 15:24:28.956352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.956026    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24" event={"ID":"15733bb9-1506-4ab3-b622-217c34be5091","Type":"ContainerDied","Data":"08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3"}
Nov 02 15:24:28.956352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.956041    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08ed0de773b4bd719e52cc02e0baa49baf38b5942f1f08423d410e4742b441c3"
Nov 02 15:24:28.956352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.956154    4470 scope.go:122] "RemoveContainer" containerID="83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"
Nov 02 15:24:28.956852 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:28.956437    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c\": container with ID starting with 83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c not found: ID does not exist" containerID="83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"
Nov 02 15:24:28.956852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.956465    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c"} err="failed to get container status \"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c\": rpc error: code = NotFound desc = could not find container \"83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c\": container with ID starting with 83b6e1047e3211f5e76157d9438578b521ef8fcc6794d798d05567f826406a7c not found: ID does not exist"
Nov 02 15:24:28.956852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.956482    4470 scope.go:122] "RemoveContainer" containerID="deb8f743eac465840bb7e0a42851dee08c4da6fde1c6d43057cea23ff484e763"
Nov 02 15:24:28.959344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.958210    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerStarted","Data":"191a0726661db43fceeb0de5d0a8d64e44cab2e9dbd5ccbcf71c89c1588699e3"}
Nov 02 15:24:28.959344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.958612    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:24:28.960787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.960729    4470 generic.go:356] "Generic (PLEG): container finished" podID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerID="5232b0e381fca051d9eb7a9f4219d24fd50633c875f953fc0c4127ddccf0fbab" exitCode=0
Nov 02 15:24:28.960787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.960770    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" event={"ID":"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7","Type":"ContainerDied","Data":"5232b0e381fca051d9eb7a9f4219d24fd50633c875f953fc0c4127ddccf0fbab"}
Nov 02 15:24:28.960787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.960802    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f" event={"ID":"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7","Type":"ContainerDied","Data":"250dd917e77e59e6e3a482695456790e7a5233cb38925a262f510903deaf7ef9"}
Nov 02 15:24:28.961001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.960821    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="250dd917e77e59e6e3a482695456790e7a5233cb38925a262f510903deaf7ef9"
Nov 02 15:24:28.963967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.963781    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:24:28.973927 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.970149    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:24:28.974139 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.973946    4470 scope.go:122] "RemoveContainer" containerID="1e9bdf4c44db504a844d430dc2789818088d47bf214ad6f6136490b4b8b4c817"
Nov 02 15:24:28.975520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.975001    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:24:28.979687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.978780    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.979687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.978811    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.979687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.978828    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcfdc\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-kube-api-access-wcfdc\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.979687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.978846    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.979687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.978862    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:28.982310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:28.981176    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:24:29.020365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.020333    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.186818    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.186874    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.186916    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.186947    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187034    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187063    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187109    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187143    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187174    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187232    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187263    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187289    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187340    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187397    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187456    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187551    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187584    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187635    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187670    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187703    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187736    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187773    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187821    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187873    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.187918    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188001    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188034    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188071    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188099    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188159    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188279    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188325    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\") pod \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\" (UID: \"629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188369    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188447    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188498    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188529    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188565    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188609    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\") pod \"10572e2d-8698-4f35-8c23-8437bbd65edb\" (UID: \"10572e2d-8698-4f35-8c23-8437bbd65edb\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188640    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188677    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188732    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188799    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188849    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\") pod \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\" (UID: \"3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188897    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\") pod \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\" (UID: \"ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4\") "
Nov 02 15:24:29.205112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.188968    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\") pod \"4641d0dc-3dca-4066-8222-3d9b59dd2658\" (UID: \"4641d0dc-3dca-4066-8222-3d9b59dd2658\") "
Nov 02 15:24:29.208003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.205844    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.209943    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.210490    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.210801    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.210809    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.211973    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212366    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212423    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212464    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212502    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212542    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.212707    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.213837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.213539    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.213888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.213841    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.215393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.214307    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.215393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.214452    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:24:29.215393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.214828    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.224377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.214710    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.326497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.230234    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.326497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.244832    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.326497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.257483    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:24:29.326497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.325656    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.329337    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.329428    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.329705    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.329892    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.329995    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.330750    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.330781    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.330962    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331505    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331575    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331641    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331687    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331748    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331806    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331911    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.331967    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332086    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332144    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332246    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332291    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332419    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332474    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332522    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332573    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332624    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332702    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332757    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332867    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332897    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.332954    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\") pod \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\" (UID: \"62ea1a00-4254-44b4-9e3d-ca1a322e89de\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.333002    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.333066    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.333167    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.333231    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\") pod \"81c60443-8c58-466a-ae68-95b76e45de67\" (UID: \"81c60443-8c58-466a-ae68-95b76e45de67\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.333293    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\") pod \"45f12615-7aa9-4452-81eb-d7122a681047\" (UID: \"45f12615-7aa9-4452-81eb-d7122a681047\") "
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334346    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334368    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334378    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334387    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334396    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334405    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10572e2d-8698-4f35-8c23-8437bbd65edb-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334414    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334424    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334439    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334451    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334461    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334470    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334479    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334488    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334497    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334512    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334522    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334533    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4641d0dc-3dca-4066-8222-3d9b59dd2658-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334542    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334552    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334562    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334571    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334583    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334596    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334604    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334613    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334621    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.334630    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.335431    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.336418    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.339775    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.340110    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.341234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.341172    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.352544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.348658    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.352544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.351877    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.352544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.352061    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:24:29.356273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.355908    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:24:29.360117 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.360093    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:24:29.360565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.360537    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.360814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.360789    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.360991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.360971    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361042    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361130 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361099    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361150    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361144    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.361465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361441    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361655    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.361874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.361854    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.362262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.362171    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.364763 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.364740    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366477    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366624    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "kube-api-access-xtwwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366738    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "kube-api-access-bcqrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366763    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "kube-api-access-jl7d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366791    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "kube-api-access-c64vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366833    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "kube-api-access-dshqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366850    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366884    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366903    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo" pod "4641d0dc-3dca-4066-8222-3d9b59dd2658" (UID: "4641d0dc-3dca-4066-8222-3d9b59dd2658"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.366960    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "kube-api-access-sdr6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.368076    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "kube-api-access-7tz6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.369101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.368356    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.370363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.370328    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.370582 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.370558    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.371261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.371228    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy" pod "10572e2d-8698-4f35-8c23-8437bbd65edb" (UID: "10572e2d-8698-4f35-8c23-8437bbd65edb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.371393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.371374    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token" pod "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" (UID: "3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.371490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.371473    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.371569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.371553    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.371761    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token" pod "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" (UID: "ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.372093    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "kube-api-access-qrwvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.372153    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.372471    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket" pod "62ea1a00-4254-44b4-9e3d-ca1a322e89de" (UID: "62ea1a00-4254-44b4-9e3d-ca1a322e89de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.372680    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert" pod "45f12615-7aa9-4452-81eb-d7122a681047" (UID: "45f12615-7aa9-4452-81eb-d7122a681047"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.373334    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.373506    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token" pod "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" (UID: "629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.380136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.379393    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo" pod "81c60443-8c58-466a-ae68-95b76e45de67" (UID: "81c60443-8c58-466a-ae68-95b76e45de67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.389587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.389445    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:24:29.392018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.391930    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:24:29.394077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.394051    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:24:29.419663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.419611    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:24:29.423914 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.423881    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-rld75"]
Nov 02 15:24:29.435776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.435730    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.435776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.435773    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.436022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.435805    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.436022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.435955    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.436133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436074    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.436133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436090    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.436279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.435987    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.436378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436354    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.436438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436365    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.436438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436426    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.436536 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436460    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.436536 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436505    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.436643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436553    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.436643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436591    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.436741 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436650    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.436741 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436703    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.436853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436746    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.436853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436774    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.436853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436844    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.437002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436893    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.437002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436922    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.437002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436965    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.437161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437016    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.437161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437052    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.437161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437084    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.437161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437136    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.437382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437167    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.437382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437216    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.437382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437273    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.437382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437304    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.437382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437348    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437408    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437443    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437492    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437527    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437560    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437588    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.437634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437619    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437651    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437679    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437725    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437757    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\") pod \"d31c3a95-47cd-4135-9009-2e8fc2320881\" (UID: \"d31c3a95-47cd-4135-9009-2e8fc2320881\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437809    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437838    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437875    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437925    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437958    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.437992    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.438039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438041    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438081    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438107    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438158    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438212    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438242    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438279    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\") pod \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\" (UID: \"8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438315    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438357    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438393    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\") pod \"4aa1e379-8d3b-4359-8fba-257fd79483c4\" (UID: \"4aa1e379-8d3b-4359-8fba-257fd79483c4\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438431    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438466    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438501    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\") pod \"f348580c-2b74-46bf-8890-e86cfbbc0c19\" (UID: \"f348580c-2b74-46bf-8890-e86cfbbc0c19\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438545    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.438632 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438590    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.439394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438642    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\") pod \"e3ed6bbc-337d-431b-bc04-1736db28c879\" (UID: \"e3ed6bbc-337d-431b-bc04-1736db28c879\") "
Nov 02 15:24:29.439394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438674    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\") pod \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\" (UID: \"9ad75510-0fdf-4413-9225-87e9b3d6bd54\") "
Nov 02 15:24:29.439394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438755    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\") pod \"15733bb9-1506-4ab3-b622-217c34be5091\" (UID: \"15733bb9-1506-4ab3-b622-217c34be5091\") "
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.436964    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439657    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439852    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439869    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439865    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439875    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439879    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439908    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439919    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl7d5\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-kube-api-access-jl7d5\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439930    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81c60443-8c58-466a-ae68-95b76e45de67-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439942    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439956    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81c60443-8c58-466a-ae68-95b76e45de67-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439970    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439979    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrwvv\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-kube-api-access-qrwvv\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440038    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438486    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440058    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440069    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45f12615-7aa9-4452-81eb-d7122a681047-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440080    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440089    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10572e2d-8698-4f35-8c23-8437bbd65edb-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440101    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440110    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81c60443-8c58-466a-ae68-95b76e45de67-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440119    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440130    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdr6m\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-kube-api-access-sdr6m\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440139    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440150    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440159    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440168    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45f12615-7aa9-4452-81eb-d7122a681047-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440178    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440203    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440217    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440232    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440241    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45f12615-7aa9-4452-81eb-d7122a681047-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440250    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtwwf\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-kube-api-access-xtwwf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440259    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440270    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c64vv\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-kube-api-access-c64vv\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440280    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440290    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440299    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f348580c-2b74-46bf-8890-e86cfbbc0c19-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440313    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.440311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440334    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440353    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcqrh\" (UniqueName: \"kubernetes.io/projected/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4-kube-api-access-bcqrh\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440367    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440383    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440397    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440412    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440426    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440439    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dshqs\" (UniqueName: \"kubernetes.io/projected/10572e2d-8698-4f35-8c23-8437bbd65edb-kube-api-access-dshqs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440489    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4641d0dc-3dca-4066-8222-3d9b59dd2658-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440507    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440523    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tz6v\" (UniqueName: \"kubernetes.io/projected/62ea1a00-4254-44b4-9e3d-ca1a322e89de-kube-api-access-7tz6v\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440538    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440554    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81c60443-8c58-466a-ae68-95b76e45de67-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440568    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440582    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45f12615-7aa9-4452-81eb-d7122a681047-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438662    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.438886    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439137    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439254    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439282    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439420    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439460    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439479    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.439588    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440088    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440406    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440685    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.440922    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.441122    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.442181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.442151    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "kube-api-access-m67mf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.442364    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.442408    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.442451    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.442983    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.443132    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.443129    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.443483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.443318    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.443677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.443492    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.443910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.443876    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.444317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444289    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.444317 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444296    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.444484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444353    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.444484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444404    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.444484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444453    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.445021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444737    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.445021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444920    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "kube-api-access-86qmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.445021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.444990    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.445249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.445047    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "kube-api-access-fsl8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.445568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.445542    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.447020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.446767    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.447020 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.446895    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "kube-api-access-jqmjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.447484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.447452    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.447768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.447696    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.447836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.447782    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "kube-api-access-96rlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.447952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.447928    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.448148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448124    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.448269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448156    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.448269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448165    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:29.448396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448343    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data" pod "15733bb9-1506-4ab3-b622-217c34be5091" (UID: "15733bb9-1506-4ab3-b622-217c34be5091"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.448449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448411    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.448670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448628    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data" pod "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" (UID: "8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.448670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448641    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.448797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.448732    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.449732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.449708    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "kube-api-access-gs578". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.450375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.450356    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy" pod "9ad75510-0fdf-4413-9225-87e9b3d6bd54" (UID: "9ad75510-0fdf-4413-9225-87e9b3d6bd54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.450713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.450672    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf" pod "d31c3a95-47cd-4135-9009-2e8fc2320881" (UID: "d31c3a95-47cd-4135-9009-2e8fc2320881"). InnerVolumeSpecName "kube-api-access-2bmnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.450896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.450879    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy" pod "e3ed6bbc-337d-431b-bc04-1736db28c879" (UID: "e3ed6bbc-337d-431b-bc04-1736db28c879"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:29.451014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.450997    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token" pod "4aa1e379-8d3b-4359-8fba-257fd79483c4" (UID: "4aa1e379-8d3b-4359-8fba-257fd79483c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:29.451326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.451293    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo" pod "f348580c-2b74-46bf-8890-e86cfbbc0c19" (UID: "f348580c-2b74-46bf-8890-e86cfbbc0c19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541637    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541669    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541679    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541689    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m67mf\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-kube-api-access-m67mf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541702    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86qmq\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-kube-api-access-86qmq\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541714    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541725    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.541727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541733    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541742    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541753    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541763    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541771    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541784    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541796    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15733bb9-1506-4ab3-b622-217c34be5091-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541806    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541818    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541830    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541843    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96rlg\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-kube-api-access-96rlg\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541856    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541872    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15733bb9-1506-4ab3-b622-217c34be5091-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541888    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541901    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541910    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541919    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541927    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bmnf\" (UniqueName: \"kubernetes.io/projected/d31c3a95-47cd-4135-9009-2e8fc2320881-kube-api-access-2bmnf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541937    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541951    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541964    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541977    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d31c3a95-47cd-4135-9009-2e8fc2320881-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541985    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d31c3a95-47cd-4135-9009-2e8fc2320881-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.541999    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542014    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542027    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542043    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542058    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4aa1e379-8d3b-4359-8fba-257fd79483c4-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542071    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542084    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3ed6bbc-337d-431b-bc04-1736db28c879-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542093    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542104    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqmjg\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-kube-api-access-jqmjg\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542120    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsl8d\" (UniqueName: \"kubernetes.io/projected/f348580c-2b74-46bf-8890-e86cfbbc0c19-kube-api-access-fsl8d\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542132    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542141    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542149    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542157    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542166    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542174    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3ed6bbc-337d-431b-bc04-1736db28c879-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542182    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542221    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f348580c-2b74-46bf-8890-e86cfbbc0c19-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542231    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542240    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542250    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs578\" (UniqueName: \"kubernetes.io/projected/e3ed6bbc-337d-431b-bc04-1736db28c879-kube-api-access-gs578\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542263    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542272    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15733bb9-1506-4ab3-b622-217c34be5091-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542288    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ad75510-0fdf-4413-9225-87e9b3d6bd54-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542299    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15733bb9-1506-4ab3-b622-217c34be5091-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542307    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542315    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.542336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.542323    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4aa1e379-8d3b-4359-8fba-257fd79483c4-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:29.710271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.710219    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" path="/var/lib/kubelet/pods/4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3/volumes"
Nov 02 15:24:29.969628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.969595    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"
Nov 02 15:24:29.972434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.972398    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerStarted","Data":"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"}
Nov 02 15:24:29.975432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.975405    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"
Nov 02 15:24:29.975595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.975411    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7" event={"ID":"f348580c-2b74-46bf-8890-e86cfbbc0c19","Type":"ContainerDied","Data":"473c1133533b58459f04020b7c8d291c25da6a9d87880fe7951a98d1fc7ded63"}
Nov 02 15:24:29.975595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.975543    4470 scope.go:122] "RemoveContainer" containerID="5496d447f6ae23f9d899b11a7081ff9c5064d3743698d94cf1ac2034f3d9869c"
Nov 02 15:24:29.977182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.977159    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"
Nov 02 15:24:29.977823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.977800    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"
Nov 02 15:24:29.978571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.978540    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"
Nov 02 15:24:29.978837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.978818    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"
Nov 02 15:24:29.978976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.978820    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"
Nov 02 15:24:29.979076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.978966    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"
Nov 02 15:24:29.979224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.979166    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"
Nov 02 15:24:29.979359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.978543    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"
Nov 02 15:24:29.980059 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.980040    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"
Nov 02 15:24:29.980183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.980153    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"
Nov 02 15:24:29.980354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.980142    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"
Nov 02 15:24:29.981007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.980985    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"
Nov 02 15:24:29.981625 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:29.981606    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"
Nov 02 15:24:30.003758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.003719    4470 patch_prober.go:29] interesting pod/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:24:30.003970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.003801    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:24:30.245560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.234825    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:30.330085 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.330052    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:30.660490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.660408    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:24:30.660490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:30.660468    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:24:31.049787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.049757    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:31.057390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.057358    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:24:31.345869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.345832    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:24:31.371155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.371121    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:24:31.375542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.375520    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:24:31.380755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.380729    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sm527"]
Nov 02 15:24:31.383684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.383658    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:24:31.395206 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395166    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.395345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395230    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.395345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395272    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.395345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395302    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395380    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395453    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395493    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395537    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395568    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395616    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395666    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395702    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395759    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395773    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395798    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395850    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395908    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395946    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.395975    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396015    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396042    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396073    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396117    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396159    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396258    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\") pod \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\" (UID: \"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396291    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396294    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396378    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\") pod \"642ce871-6b62-4d7d-9fc9-2df3207a499c\" (UID: \"642ce871-6b62-4d7d-9fc9-2df3207a499c\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396447    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\") pod \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\" (UID: \"b716ef3d-edfe-4b55-b337-18e87e77d4ae\") "
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396546    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.396857    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.397405    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.398120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398068    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398270    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398424    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398474    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398495    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398512    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398531    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398549    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398565    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398580    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398750    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398836    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.398928    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.399139    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.399419    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.399919    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400061    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400300    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:31.400595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400562    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.401523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400675    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.401523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400818    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.401523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.400965    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:31.401686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.401536    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j" pod "b716ef3d-edfe-4b55-b337-18e87e77d4ae" (UID: "b716ef3d-edfe-4b55-b337-18e87e77d4ae"). InnerVolumeSpecName "kube-api-access-jl47j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.401686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.401632    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "kube-api-access-85298". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.401834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.401722    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "kube-api-access-nksds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.401834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.401748    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.402101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.402073    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.402273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.402108    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy" pod "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" (UID: "a79d6e7a-b8f0-41bc-af9c-03b490c6ae19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.403476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.403451    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token" pod "642ce871-6b62-4d7d-9fc9-2df3207a499c" (UID: "642ce871-6b62-4d7d-9fc9-2df3207a499c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505084    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505130    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505149    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505167    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl47j\" (UniqueName: \"kubernetes.io/projected/b716ef3d-edfe-4b55-b337-18e87e77d4ae-kube-api-access-jl47j\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505182    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505213    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b716ef3d-edfe-4b55-b337-18e87e77d4ae-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505227    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505245    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505259    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/642ce871-6b62-4d7d-9fc9-2df3207a499c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505278    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505291    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b716ef3d-edfe-4b55-b337-18e87e77d4ae-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505304    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85298\" (UniqueName: \"kubernetes.io/projected/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-kube-api-access-85298\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505319    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505335    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505348    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505361    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505375    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/642ce871-6b62-4d7d-9fc9-2df3207a499c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505391    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505406    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nksds\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-kube-api-access-nksds\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.505502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.505421    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/642ce871-6b62-4d7d-9fc9-2df3207a499c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.513339 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.513215    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:24:31.548461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.548373    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-g87qp"]
Nov 02 15:24:31.691378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.691298    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:24:31.722733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.722702    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" path="/var/lib/kubelet/pods/4aa1e379-8d3b-4359-8fba-257fd79483c4/volumes"
Nov 02 15:24:31.723274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.723258    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" path="/var/lib/kubelet/pods/ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4/volumes"
Nov 02 15:24:31.724051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.724032    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-jx4f7"]
Nov 02 15:24:31.729572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.729543    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:24:31.791835 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.791742    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:24:31.796220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.796164    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-zft9f"]
Nov 02 15:24:31.812402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812366    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812472    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812517    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812537    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812590    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812636    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812718    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812825    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.812871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812821    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.812871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812843    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.813019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812857    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\") pod \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\" (UID: \"4d6d5321-f8fc-4d72-a2a9-28d2eef87979\") "
Nov 02 15:24:31.813019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.812907    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.813134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813059    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.813183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813140    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:24:31.813668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813641    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.813766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813674    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.813766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813693    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.813766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813710    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.813766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.813725    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.814759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.814738    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.814934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.814920    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "kube-api-access-rs5w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:24:31.815242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.815222    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:24:31.815302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.815268    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo" pod "4d6d5321-f8fc-4d72-a2a9-28d2eef87979" (UID: "4d6d5321-f8fc-4d72-a2a9-28d2eef87979"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:24:31.915014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.914962    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.915014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.915002    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.915014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.915020    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.915447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.915035    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs5w2\" (UniqueName: \"kubernetes.io/projected/4d6d5321-f8fc-4d72-a2a9-28d2eef87979-kube-api-access-rs5w2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:24:31.915905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.915843    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:24:31.919522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:31.919489    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-hrf24"]
Nov 02 15:24:32.043959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.043852    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podStartSLOduration=6.043837684 podStartE2EDuration="6.043837684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:32.040777651 +0000 UTC m=+1678.478679794" watchObservedRunningTime="2026-11-02 15:24:32.043837684 +0000 UTC m=+1678.481739824"
Nov 02 15:24:32.059370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.059332    4470 generic.go:356] "Generic (PLEG): container finished" podID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerID="651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c" exitCode=0
Nov 02 15:24:32.059842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.059426    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"
Nov 02 15:24:32.059842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.059436    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" event={"ID":"4d6d5321-f8fc-4d72-a2a9-28d2eef87979","Type":"ContainerDied","Data":"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"}
Nov 02 15:24:32.059842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.059586    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" event={"ID":"4d6d5321-f8fc-4d72-a2a9-28d2eef87979","Type":"ContainerDied","Data":"46771254711f2715dfba6679649933941a8e668461fae565b4e06c1fda2c44f5"}
Nov 02 15:24:32.059842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.059641    4470 scope.go:122] "RemoveContainer" containerID="651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"
Nov 02 15:24:32.063613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.063581    4470 generic.go:356] "Generic (PLEG): container finished" podID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerID="68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3" exitCode=0
Nov 02 15:24:32.063780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.063661    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerDied","Data":"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"}
Nov 02 15:24:32.063780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.063673    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"
Nov 02 15:24:32.063780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.063718    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb" event={"ID":"a79d6e7a-b8f0-41bc-af9c-03b490c6ae19","Type":"ContainerDied","Data":"34eada36b4e3777b113ec3882d0515acd224eae2f3d66669ba683068f75f2602"}
Nov 02 15:24:32.066924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.066894    4470 generic.go:356] "Generic (PLEG): container finished" podID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerID="a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e" exitCode=0
Nov 02 15:24:32.068530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.066984    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" event={"ID":"b716ef3d-edfe-4b55-b337-18e87e77d4ae","Type":"ContainerDied","Data":"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"}
Nov 02 15:24:32.068530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.066995    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"
Nov 02 15:24:32.068530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.067459    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq" event={"ID":"b716ef3d-edfe-4b55-b337-18e87e77d4ae","Type":"ContainerDied","Data":"3a0bcb72fee7936e1e4321ef7117e44a42afe0ead2ff1f1fdb1fea731f0782f0"}
Nov 02 15:24:32.070527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.070311    4470 generic.go:356] "Generic (PLEG): container finished" podID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerID="1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323" exitCode=0
Nov 02 15:24:32.071219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.071199    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"
Nov 02 15:24:32.071665 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.071647    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" event={"ID":"642ce871-6b62-4d7d-9fc9-2df3207a499c","Type":"ContainerDied","Data":"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"}
Nov 02 15:24:32.071745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.071687    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b" event={"ID":"642ce871-6b62-4d7d-9fc9-2df3207a499c","Type":"ContainerDied","Data":"8f29bd5e08de90795b7e17d858888369dfb546fa63ab203dd3ccd8f2bd12021e"}
Nov 02 15:24:32.076344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.076169    4470 scope.go:122] "RemoveContainer" containerID="651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"
Nov 02 15:24:32.076635 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:32.076611    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c\": container with ID starting with 651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c not found: ID does not exist" containerID="651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"
Nov 02 15:24:32.076724 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.076649    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c"} err="failed to get container status \"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c\": rpc error: code = NotFound desc = could not find container \"651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c\": container with ID starting with 651ff71316b5e7cbf06f28fb60c1167e92b90b62be20cb791bd0c8dff6a8b00c not found: ID does not exist"
Nov 02 15:24:32.076724 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.076674    4470 scope.go:122] "RemoveContainer" containerID="68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"
Nov 02 15:24:32.092950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.092921    4470 scope.go:122] "RemoveContainer" containerID="1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"
Nov 02 15:24:32.104180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.104150    4470 scope.go:122] "RemoveContainer" containerID="68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"
Nov 02 15:24:32.104597 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:32.104568    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3\": container with ID starting with 68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3 not found: ID does not exist" containerID="68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"
Nov 02 15:24:32.104684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.104604    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3"} err="failed to get container status \"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3\": rpc error: code = NotFound desc = could not find container \"68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3\": container with ID starting with 68b6b2a5a33b91ba01b1ec0b3b808655c7759ec3c96713e9ff3bc68eadd34aa3 not found: ID does not exist"
Nov 02 15:24:32.104684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.104624    4470 scope.go:122] "RemoveContainer" containerID="1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"
Nov 02 15:24:32.104951 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:32.104920    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff\": container with ID starting with 1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff not found: ID does not exist" containerID="1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"
Nov 02 15:24:32.105045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.104956    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff"} err="failed to get container status \"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff\": rpc error: code = NotFound desc = could not find container \"1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff\": container with ID starting with 1624dc8a3c51d739c686d75e19454f502751ae0c1b0784a587c1330fb820e7ff not found: ID does not exist"
Nov 02 15:24:32.105045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.104979    4470 scope.go:122] "RemoveContainer" containerID="a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"
Nov 02 15:24:32.115011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.114988    4470 scope.go:122] "RemoveContainer" containerID="a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"
Nov 02 15:24:32.115339 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:32.115317    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e\": container with ID starting with a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e not found: ID does not exist" containerID="a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"
Nov 02 15:24:32.115403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.115347    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e"} err="failed to get container status \"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e\": rpc error: code = NotFound desc = could not find container \"a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e\": container with ID starting with a1f58b991cb761377a09bc0640c1b8940fefe35e0635655ecb04ae00a20fdd0e not found: ID does not exist"
Nov 02 15:24:32.115403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.115366    4470 scope.go:122] "RemoveContainer" containerID="1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"
Nov 02 15:24:32.125124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.125102    4470 scope.go:122] "RemoveContainer" containerID="1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"
Nov 02 15:24:32.125479 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:32.125456    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323\": container with ID starting with 1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323 not found: ID does not exist" containerID="1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"
Nov 02 15:24:32.125568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.125486    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323"} err="failed to get container status \"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323\": rpc error: code = NotFound desc = could not find container \"1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323\": container with ID starting with 1f3c6c7cea963b82488f5e1ec3c3f9c8b8ffd09703e8d86c7990e05c47cea323 not found: ID does not exist"
Nov 02 15:24:32.183164 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.183051    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:24:32.188068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.188017    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-6dc69b-openshift-default-b67c99fc7-kg77j"]
Nov 02 15:24:32.293311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.293276    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:24:32.299572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.299412    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-sdfkt"]
Nov 02 15:24:32.407094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.407060    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:24:32.410400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.410380    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-qzftv"]
Nov 02 15:24:32.512992 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.512949    4470 patch_prober.go:29] interesting pod/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:24:32.513151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.513015    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:24:32.531823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.531783    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:24:32.537537 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.537505    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-pcpdh"]
Nov 02 15:24:32.652829 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.652739    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:24:32.656124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.656097    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-db3931-openshift-default-5f47d69f9f-nn6ft"]
Nov 02 15:24:32.799868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.799823    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:24:32.803387 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.803359    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-z6pj7"]
Nov 02 15:24:32.898947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.898914    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:24:32.902220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:32.902182    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-fbzg6"]
Nov 02 15:24:33.007850 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.007813    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:24:33.014814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.014785    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-019efe-openshift-default-5d87cb7859-4vqq6"]
Nov 02 15:24:33.125199 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.125165    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:24:33.132305 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.132276    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-v2c4f"]
Nov 02 15:24:33.255297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.255262    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:24:33.261887 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.261808    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ba49d0-openshift-default-6b978b7557-kncw4"]
Nov 02 15:24:33.461985 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.461940    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:24:33.464760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.464734    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-lqxzq"]
Nov 02 15:24:33.567553 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:24:33.567454    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:24:33.649888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.649848    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:24:33.657337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.657309    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-n8ct7"]
Nov 02 15:24:33.716465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.716432    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" path="/var/lib/kubelet/pods/10572e2d-8698-4f35-8c23-8437bbd65edb/volumes"
Nov 02 15:24:33.716957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.716944    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15733bb9-1506-4ab3-b622-217c34be5091" path="/var/lib/kubelet/pods/15733bb9-1506-4ab3-b622-217c34be5091/volumes"
Nov 02 15:24:33.717450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.717435    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" path="/var/lib/kubelet/pods/3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784/volumes"
Nov 02 15:24:33.718212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.718179    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45f12615-7aa9-4452-81eb-d7122a681047" path="/var/lib/kubelet/pods/45f12615-7aa9-4452-81eb-d7122a681047/volumes"
Nov 02 15:24:33.718655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.718639    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" path="/var/lib/kubelet/pods/4641d0dc-3dca-4066-8222-3d9b59dd2658/volumes"
Nov 02 15:24:33.719076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.719063    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" path="/var/lib/kubelet/pods/4d6d5321-f8fc-4d72-a2a9-28d2eef87979/volumes"
Nov 02 15:24:33.719791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.719775    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" path="/var/lib/kubelet/pods/629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7/volumes"
Nov 02 15:24:33.720218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.720202    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" path="/var/lib/kubelet/pods/62ea1a00-4254-44b4-9e3d-ca1a322e89de/volumes"
Nov 02 15:24:33.720913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.720898    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81c60443-8c58-466a-ae68-95b76e45de67" path="/var/lib/kubelet/pods/81c60443-8c58-466a-ae68-95b76e45de67/volumes"
Nov 02 15:24:33.721367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.721354    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" path="/var/lib/kubelet/pods/8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17/volumes"
Nov 02 15:24:33.721773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.721760    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" path="/var/lib/kubelet/pods/9ad75510-0fdf-4413-9225-87e9b3d6bd54/volumes"
Nov 02 15:24:33.722465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.722451    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" path="/var/lib/kubelet/pods/b716ef3d-edfe-4b55-b337-18e87e77d4ae/volumes"
Nov 02 15:24:33.722897 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.722883    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" path="/var/lib/kubelet/pods/d31c3a95-47cd-4135-9009-2e8fc2320881/volumes"
Nov 02 15:24:33.723327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.723314    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" path="/var/lib/kubelet/pods/e3ed6bbc-337d-431b-bc04-1736db28c879/volumes"
Nov 02 15:24:33.724021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.724000    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" path="/var/lib/kubelet/pods/f348580c-2b74-46bf-8890-e86cfbbc0c19/volumes"
Nov 02 15:24:33.856311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.856216    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:24:33.862484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:33.862396    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-ngxtb"]
Nov 02 15:24:34.057709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.057676    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:24:34.063373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.063337    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-c18e79-openshift-default-59544557d7-p4t2b"]
Nov 02 15:24:34.102830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.096411    4470 scope.go:122] "RemoveContainer" containerID="a97ab39269b2a1ea61a8bee2a9a6ffa01f30cc7d8d5c86f058e4cc22065b8873"
Nov 02 15:24:34.118113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.118093    4470 scope.go:122] "RemoveContainer" containerID="d069bf55a2604e9f00f408f56096080c4a56f59a5adde685dda533bc39865147"
Nov 02 15:24:34.128045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.128016    4470 scope.go:122] "RemoveContainer" containerID="af23046874219a126da818ddb3143eba7c7ab4bccc9435c4770af511f886b989"
Nov 02 15:24:34.138818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.138789    4470 scope.go:122] "RemoveContainer" containerID="22ae92078e07bfc3def4e6d735d80b56f8ce6ba37e216e3a63851fa1dd42ed96"
Nov 02 15:24:34.149375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.149356    4470 scope.go:122] "RemoveContainer" containerID="1e81b4a0a95fd3d21164e1ba94e40bd39576be5c3870587822ae2a26f09ff662"
Nov 02 15:24:34.158823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.158799    4470 scope.go:122] "RemoveContainer" containerID="190efd00b2fff32c1cf328bd363fb6e0b0aca6baf6912941c5c262124d2bbf90"
Nov 02 15:24:34.168869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.168849    4470 scope.go:122] "RemoveContainer" containerID="b58fe1f21700dbe6c5cea687cda1b749e4cd49d8d7f8afcdc22e98947050b219"
Nov 02 15:24:34.205464 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.205440    4470 scope.go:122] "RemoveContainer" containerID="16ab0570d68f48dfc2508d83e562ee0d55f26fca8cfbdd0a2e621a568447568d"
Nov 02 15:24:34.216793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.216772    4470 scope.go:122] "RemoveContainer" containerID="0b2e054a6e35f55d8758645cdd3e6b1a5b3adc243070918e7be800e72e067ff9"
Nov 02 15:24:34.226874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.226855    4470 scope.go:122] "RemoveContainer" containerID="5232b0e381fca051d9eb7a9f4219d24fd50633c875f953fc0c4127ddccf0fbab"
Nov 02 15:24:34.236532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.236513    4470 scope.go:122] "RemoveContainer" containerID="c35baba076ca88fcf4a32a2eb96269366d9cbc3934e0cf22624f4568679bf078"
Nov 02 15:24:34.246172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:34.246151    4470 scope.go:122] "RemoveContainer" containerID="b05236150165be48ae5cfb5b99479f708facd6e634e911aa5dd314b3df0f65ca"
Nov 02 15:24:35.706935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:35.706892    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" path="/var/lib/kubelet/pods/642ce871-6b62-4d7d-9fc9-2df3207a499c/volumes"
Nov 02 15:24:35.707662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:35.707641    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" path="/var/lib/kubelet/pods/a79d6e7a-b8f0-41bc-af9c-03b490c6ae19/volumes"
Nov 02 15:24:40.375266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.373864    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:24:40.375266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.373982    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.698344    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699303    4470 cpu_manager.go:414] "containerMap: removing container" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699326    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699348    4470 cpu_manager.go:414] "containerMap: removing container" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699356    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699369    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699377    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699390    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699396    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699407    4470 cpu_manager.go:414] "containerMap: removing container" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699414    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699423    4470 cpu_manager.go:414] "containerMap: removing container" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699429    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699440    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699448    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699461    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699469    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699486    4470 cpu_manager.go:414] "containerMap: removing container" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699492    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699500    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699507    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699525    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699531    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699541    4470 cpu_manager.go:414] "containerMap: removing container" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699547    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699558    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699564    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699575    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699582    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699591    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699598    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699630    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699638    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699651    4470 cpu_manager.go:414] "containerMap: removing container" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699658    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699668    4470 cpu_manager.go:414] "containerMap: removing container" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699674    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699685    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699691    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699709    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699716    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699726    4470 cpu_manager.go:414] "containerMap: removing container" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699732    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699742    4470 cpu_manager.go:414] "containerMap: removing container" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699748    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699761    4470 cpu_manager.go:414] "containerMap: removing container" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699768    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699908    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699919    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="45f12615-7aa9-4452-81eb-d7122a681047" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699929    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ad75510-0fdf-4413-9225-87e9b3d6bd54" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699938    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699948    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d31c3a95-47cd-4135-9009-2e8fc2320881" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699956    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4641d0dc-3dca-4066-8222-3d9b59dd2658" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699966    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="642ce871-6b62-4d7d-9fc9-2df3207a499c" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.699973    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="62ea1a00-4254-44b4-9e3d-ca1a322e89de" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700005    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a79d6e7a-b8f0-41bc-af9c-03b490c6ae19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700014    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aa1e379-8d3b-4359-8fba-257fd79483c4" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700023    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="b716ef3d-edfe-4b55-b337-18e87e77d4ae" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700031    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="81c60443-8c58-466a-ae68-95b76e45de67" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700039    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="629fce1e-adfd-4bb5-ac9d-bdeec85ba6a7" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700049    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="f348580c-2b74-46bf-8890-e86cfbbc0c19" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700058    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700065    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d6d5321-f8fc-4d72-a2a9-28d2eef87979" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700073    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4eb4dff8-15f7-4c63-9bef-2ef9b7cbcdc3" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700081    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="15733bb9-1506-4ab3-b622-217c34be5091" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700090    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="10572e2d-8698-4f35-8c23-8437bbd65edb" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700098    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3ed6bbc-337d-431b-bc04-1736db28c879" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700107    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cffb1ec-3c2e-4c1b-a502-938b3b1a0d17" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700116    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b4e512f-b6a5-4a21-8bf8-b4b02bbcc784" containerName="istio-proxy"
Nov 02 15:24:40.709211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.700124    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffc7a91a-d2ce-4f6c-a932-0ecf39f185a4" containerName="istio-proxy"
Nov 02 15:24:40.719258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.717071    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.817726    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838405    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838491    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838525    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb7vr\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838589    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838697    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838733    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838767    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.838820    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.843213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.819244    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-4c5b4e-openshift-default-dockercfg-6p2dd\""
Nov 02 15:24:40.878338 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.878289    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940073    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940179    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940244    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940273    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb7vr\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940327    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940410    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940437    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940474    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.940521    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.941034    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.956937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.946560    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.956937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.946915    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.956937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.947226    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.956937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.948066    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.956937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.948383    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:40.961796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:40.961554    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:41.023251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.023171    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:41.023527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.023436    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb7vr\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\") pod \"e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") " pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:41.137649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.137247    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:41.674135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.674084    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:24:41.677747 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:24:41.677719    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8938a9_ac6f_4754_95bf_8ffc8dec5d23.slice/crio-9a8b4a57e4f1e085c9bcf3235a3eb910509cf0b79b07fbc9c5c88de8d94551d9 WatchSource:0}: Error finding container 9a8b4a57e4f1e085c9bcf3235a3eb910509cf0b79b07fbc9c5c88de8d94551d9: Status 404 returned error can't find the container with id 9a8b4a57e4f1e085c9bcf3235a3eb910509cf0b79b07fbc9c5c88de8d94551d9
Nov 02 15:24:41.680238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.680182    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:41.680335 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.680276    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:41.680335 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:41.680303    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:42.240064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:42.240022    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerStarted","Data":"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"}
Nov 02 15:24:42.240293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:42.240084    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerStarted","Data":"9a8b4a57e4f1e085c9bcf3235a3eb910509cf0b79b07fbc9c5c88de8d94551d9"}
Nov 02 15:24:42.366230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:42.366143    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podStartSLOduration=2.366120553 podStartE2EDuration="2.366120553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:42.362926359 +0000 UTC m=+1688.800828511" watchObservedRunningTime="2026-11-02 15:24:42.366120553 +0000 UTC m=+1688.804022697"
Nov 02 15:24:43.138576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.138540    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:43.145046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.145011    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:43.249104 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.249073    4470 generic.go:356] "Generic (PLEG): container finished" podID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerID="aa2c3012d48e234f7d157fc0b042c19f5bdd9654a4c602d9b748c366ce644d22" exitCode=0
Nov 02 15:24:43.249299 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.249214    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerDied","Data":"aa2c3012d48e234f7d157fc0b042c19f5bdd9654a4c602d9b748c366ce644d22"}
Nov 02 15:24:43.249488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.249471    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:43.249806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.249789    4470 scope.go:122] "RemoveContainer" containerID="aa2c3012d48e234f7d157fc0b042c19f5bdd9654a4c602d9b748c366ce644d22"
Nov 02 15:24:43.250762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.250566    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:24:43.251392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.251367    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:43.251448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.251435    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:43.251488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:43.251468    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:24:44.257271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:44.257224    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerStarted","Data":"1e09c0e72cbfcbb03dcd4959151f67666cdab218347c8a9adaeafaabecdd75df"}
Nov 02 15:24:45.097892 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.097852    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:24:45.098129 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.098080    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 02 15:24:45.098239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.098134    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 02 15:24:45.777214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.774079    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:24:45.835213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.823459    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:45.856226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.852215    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:24:45.906214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.903926    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:24:45.919220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.916950    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-fd1339\""
Nov 02 15:24:45.935214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.927865    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzt7p\" (UniqueName: \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:45.935214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:45.928269    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.036412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.031251    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.036412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.031525    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzt7p\" (UniqueName: \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.036412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.032931    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.065216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.064718    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:24:46.067381 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.067348    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.099042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.098674    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 02 15:24:46.099042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.098745    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 02 15:24:46.118128 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.118094    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzt7p\" (UniqueName: \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\") pod \"payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.121586 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.121540    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-fd1339-dockercfg-bvb9q\""
Nov 02 15:24:46.132509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.132479    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs7s\" (UniqueName: \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.132675 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.132538    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.233590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.233551    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs7s\" (UniqueName: \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.233828 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.233603    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.234359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.234338    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.299026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.298933    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs7s\" (UniqueName: \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\") pod \"payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.338868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.338833    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:46.443499 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.443466    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:46.746842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.746724    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:24:46.749475 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:24:46.749442    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034816b3_6329_41cf_a182_19a9c91a2139.slice/crio-3b10e46127b0deeb4b61abbc5a26189cd0e3cded214fe4661d0c8892747ef9f4 WatchSource:0}: Error finding container 3b10e46127b0deeb4b61abbc5a26189cd0e3cded214fe4661d0c8892747ef9f4: Status 404 returned error can't find the container with id 3b10e46127b0deeb4b61abbc5a26189cd0e3cded214fe4661d0c8892747ef9f4
Nov 02 15:24:46.859999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:46.859975    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:24:46.860963 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:24:46.860938    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fe728c_4705_4c04_beca_56bdcb00d15e.slice/crio-fbc955193957129753f526a728b61ee92bc1d918c5587cb9661a1e224b98dd7b WatchSource:0}: Error finding container fbc955193957129753f526a728b61ee92bc1d918c5587cb9661a1e224b98dd7b: Status 404 returned error can't find the container with id fbc955193957129753f526a728b61ee92bc1d918c5587cb9661a1e224b98dd7b
Nov 02 15:24:47.098235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.098169    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 02 15:24:47.098435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.098244    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 02 15:24:47.307844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.307736    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" event={"ID":"034816b3-6329-41cf-a182-19a9c91a2139","Type":"ContainerStarted","Data":"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"}
Nov 02 15:24:47.307844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.307807    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" event={"ID":"034816b3-6329-41cf-a182-19a9c91a2139","Type":"ContainerStarted","Data":"3b10e46127b0deeb4b61abbc5a26189cd0e3cded214fe4661d0c8892747ef9f4"}
Nov 02 15:24:47.308128 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.308111    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:47.309729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.309699    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" event={"ID":"d9fe728c-4705-4c04-beca-56bdcb00d15e","Type":"ContainerStarted","Data":"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"}
Nov 02 15:24:47.309729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.309736    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" event={"ID":"d9fe728c-4705-4c04-beca-56bdcb00d15e","Type":"ContainerStarted","Data":"fbc955193957129753f526a728b61ee92bc1d918c5587cb9661a1e224b98dd7b"}
Nov 02 15:24:47.309891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.309831    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:47.589863 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.589674    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" podStartSLOduration=2.589649751 podStartE2EDuration="2.589649751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:47.449482868 +0000 UTC m=+1693.887385010" watchObservedRunningTime="2026-11-02 15:24:47.589649751 +0000 UTC m=+1694.027551893"
Nov 02 15:24:47.614004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:47.613937    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" podStartSLOduration=2.613913949 podStartE2EDuration="2.613913949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:47.582669634 +0000 UTC m=+1694.020571776" watchObservedRunningTime="2026-11-02 15:24:47.613913949 +0000 UTC m=+1694.051816092"
Nov 02 15:24:48.097826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:48.097790    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 02 15:24:48.098400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:48.097847    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 02 15:24:49.097478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:49.097439    4470 patch_prober.go:29] interesting pod/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused" start-of-body=
Nov 02 15:24:49.097793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:49.097758    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.99:15021/healthz/ready\": dial tcp 10.133.0.99:15021: connect: connection refused"
Nov 02 15:24:50.102300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.102268    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:24:50.102807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.102668    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:24:50.103468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.103449    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:24:50.904155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.904121    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:24:50.909060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.909000    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:50.959732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.959697    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:24:50.961717 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.961693    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-4c5b4e\""
Nov 02 15:24:50.961856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:50.961817    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-4c5b4e-dockercfg-hxccr\""
Nov 02 15:24:51.120168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.120126    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c79hv\" (UniqueName: \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.120780 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.120750    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.203265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.195743    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:24:51.208848 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.208815    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:24:51.209001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.208932    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.222793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.222757    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c79hv\" (UniqueName: \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.222793 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.222808    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.223719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.223693    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.288311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.288271    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c79hv\" (UniqueName: \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\") pod \"payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.324237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.324203    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhz7c\" (UniqueName: \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.324414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.324299    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.426603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.426563    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhz7c\" (UniqueName: \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.426805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.426624    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.427421 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.427398    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.487379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.487310    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhz7c\" (UniqueName: \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\") pod \"payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.527962 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.527923    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:51.540164 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.540139    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:51.925689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.925658    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:24:51.927521 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:24:51.927491    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce021ff6_36f8_4e77_83f9_8c0c2ce99dd6.slice/crio-396a1cc41640f5fe99ebd1031b0a48838583dc37c5fb95d8e5a07eb56ce10947 WatchSource:0}: Error finding container 396a1cc41640f5fe99ebd1031b0a48838583dc37c5fb95d8e5a07eb56ce10947: Status 404 returned error can't find the container with id 396a1cc41640f5fe99ebd1031b0a48838583dc37c5fb95d8e5a07eb56ce10947
Nov 02 15:24:51.974260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:51.974230    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:24:51.975882 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:24:51.975848    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d28652_3d16_49be_8051_4a29139033e1.slice/crio-5a6b73fa9a22f51eec6d97f9716a253c646e4248fafa466b061f1644abbce9e2 WatchSource:0}: Error finding container 5a6b73fa9a22f51eec6d97f9716a253c646e4248fafa466b061f1644abbce9e2: Status 404 returned error can't find the container with id 5a6b73fa9a22f51eec6d97f9716a253c646e4248fafa466b061f1644abbce9e2
Nov 02 15:24:52.346898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.346861    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" event={"ID":"c1d28652-3d16-49be-8051-4a29139033e1","Type":"ContainerStarted","Data":"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"}
Nov 02 15:24:52.347321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.346919    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" event={"ID":"c1d28652-3d16-49be-8051-4a29139033e1","Type":"ContainerStarted","Data":"5a6b73fa9a22f51eec6d97f9716a253c646e4248fafa466b061f1644abbce9e2"}
Nov 02 15:24:52.347796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.347775    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:24:52.349604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.349577    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" event={"ID":"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6","Type":"ContainerStarted","Data":"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"}
Nov 02 15:24:52.349744 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.349626    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" event={"ID":"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6","Type":"ContainerStarted","Data":"396a1cc41640f5fe99ebd1031b0a48838583dc37c5fb95d8e5a07eb56ce10947"}
Nov 02 15:24:52.349744 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.349669    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:24:52.464667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:52.464602    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" podStartSLOduration=2.464583459 podStartE2EDuration="2.464583459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:52.463343268 +0000 UTC m=+1698.901245426" watchObservedRunningTime="2026-11-02 15:24:52.464583459 +0000 UTC m=+1698.902485601"
Nov 02 15:24:58.319167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:58.319131    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:24:58.319596 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:58.319333    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:24:58.438410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:24:58.438357    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" podStartSLOduration=8.438343643 podStartE2EDuration="8.438343643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:24:52.603204016 +0000 UTC m=+1699.041106152" watchObservedRunningTime="2026-11-02 15:24:58.438343643 +0000 UTC m=+1704.876245783"
Nov 02 15:25:01.791816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:01.791781    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:25:01.792295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:01.791980    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerName="maas-api" containerID="cri-o://897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04" gracePeriod=30
Nov 02 15:25:02.160276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.160243    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:25:02.281096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.280992    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\") pod \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") "
Nov 02 15:25:02.281330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.281236    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\") pod \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") "
Nov 02 15:25:02.281401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.281391    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") pod \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") "
Nov 02 15:25:02.281541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.281477    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\" (UniqueName: \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\") pod \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\" (UID: \"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1\") "
Nov 02 15:25:02.281824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.281470    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca" pod "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" (UID: "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:25:02.282044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.282023    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:25:02.288721 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.288680    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls" pod "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" (UID: "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:25:02.288721 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.288711    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs" pod "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" (UID: "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:25:02.288947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.288728    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf" pod "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" (UID: "a47637d2-49bf-4fe8-a5f3-7d19d734c8b1"). InnerVolumeSpecName "kube-api-access-9jngf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:25:02.383690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.383608    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:25:02.383690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.383637    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:25:02.383690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.383650    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jngf\" (UniqueName: \"kubernetes.io/projected/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1-kube-api-access-9jngf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:25:02.414275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.414240    4470 generic.go:356] "Generic (PLEG): container finished" podID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerID="897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04" exitCode=0
Nov 02 15:25:02.414456 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.414293    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"
Nov 02 15:25:02.414456 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.414335    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" event={"ID":"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1","Type":"ContainerDied","Data":"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"}
Nov 02 15:25:02.414456 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.414400    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" event={"ID":"a47637d2-49bf-4fe8-a5f3-7d19d734c8b1","Type":"ContainerDied","Data":"4eee62c10a9c7bc80679f3c22e8335289e632c5d6af3a6b9c919ca2f889c7c45"}
Nov 02 15:25:02.414456 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.414424    4470 scope.go:122] "RemoveContainer" containerID="897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"
Nov 02 15:25:02.425289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.425269    4470 scope.go:122] "RemoveContainer" containerID="897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"
Nov 02 15:25:02.425622 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:02.425603    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04\": container with ID starting with 897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04 not found: ID does not exist" containerID="897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"
Nov 02 15:25:02.425678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.425633    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04"} err="failed to get container status \"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04\": rpc error: code = NotFound desc = could not find container \"897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04\": container with ID starting with 897a531c32c4fc2f081be5ea35887c4b76ea74a2c5425b9f6ff32d5bf9d55e04 not found: ID does not exist"
Nov 02 15:25:02.583467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.583147    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:25:02.586490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:02.586464    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq"]
Nov 02 15:25:03.360083 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:03.360055    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:25:03.566611 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:03.566569    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:25:03.707424 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:03.707392    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" path="/var/lib/kubelet/pods/a47637d2-49bf-4fe8-a5f3-7d19d734c8b1/volumes"
Nov 02 15:25:04.368551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:04.368523    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:25:05.025344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:05.025293    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be5d503e-6bff8f9d67-vpmgq" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.100:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:25:06.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:06.047332    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-fxb56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:25:06.048217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:06.047412    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-fxb56" podUID="acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:25:08.622964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.622925    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:25:08.623590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.623576    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerName="maas-api"
Nov 02 15:25:08.623635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.623591    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerName="maas-api"
Nov 02 15:25:08.623687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.623678    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a47637d2-49bf-4fe8-a5f3-7d19d734c8b1" containerName="maas-api"
Nov 02 15:25:08.628603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.628581    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.700730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.700685    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nc8n\" (UniqueName: \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.700938 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.700917    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.701047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.701028    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.701086 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.701071    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.734065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.734003    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-559973-serving-cert\""
Nov 02 15:25:08.740100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.740058    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:25:08.802846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.802801    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.803047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.802891    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.803047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.802924    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.803047 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:08.802968    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-serving-cert: secret "maas-api-e2e-shared-a-w2-559973-serving-cert" not found
Nov 02 15:25:08.803281 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:08.803061    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls podName:9979a001-a253-40e5-a602-8dd87cbd40b5 nodeName:}" failed. No retries permitted until 2026-11-02 15:25:09.303038483 +0000 UTC m=+1715.740940623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls") pod "maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" (UID: "9979a001-a253-40e5-a602-8dd87cbd40b5") : secret "maas-api-e2e-shared-a-w2-559973-serving-cert" not found
Nov 02 15:25:08.803281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.802978    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nc8n\" (UniqueName: \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.803772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.803752    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.805517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.805492    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:08.881750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:08.881667    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nc8n\" (UniqueName: \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:09.323051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:09.323002    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:09.325544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:09.325511    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:09.550837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:09.550801    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:09.938710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:09.938672    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:25:09.940236 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:09.940204    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9979a001_a253_40e5_a602_8dd87cbd40b5.slice/crio-ff6fe51203764895eafc813cdf9f83212fb72deee24fc5ec9181a7557fe4ca1a WatchSource:0}: Error finding container ff6fe51203764895eafc813cdf9f83212fb72deee24fc5ec9181a7557fe4ca1a: Status 404 returned error can't find the container with id ff6fe51203764895eafc813cdf9f83212fb72deee24fc5ec9181a7557fe4ca1a
Nov 02 15:25:10.484486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:10.484445    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" event={"ID":"9979a001-a253-40e5-a602-8dd87cbd40b5","Type":"ContainerStarted","Data":"ff6fe51203764895eafc813cdf9f83212fb72deee24fc5ec9181a7557fe4ca1a"}
Nov 02 15:25:11.496155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:11.496104    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" event={"ID":"9979a001-a253-40e5-a602-8dd87cbd40b5","Type":"ContainerStarted","Data":"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"}
Nov 02 15:25:11.496693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:11.496349    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:11.617060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:11.616999    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" podStartSLOduration=2.615516935 podStartE2EDuration="3.616982087s" totalImagesPullingTime="1.001465152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:11.614269317 +0000 UTC m=+1718.052171480" watchObservedRunningTime="2026-11-02 15:25:11.616982087 +0000 UTC m=+1718.054884228"
Nov 02 15:25:13.606874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.606842    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:25:13.611133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.611113    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.690888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.690843    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.691072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.690915    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.691072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.691017    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.691072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.691046    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdw2\" (UniqueName: \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.711522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.711490    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-6b2f61-serving-cert\""
Nov 02 15:25:13.714792 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.714765    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:25:13.792034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.791999    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.792253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.792099    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.792713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.792689    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.792813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.792736    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdw2\" (UniqueName: \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.792877 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:13.792811    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-serving-cert: secret "maas-api-e2e-shared-a-w5-6b2f61-serving-cert" not found
Nov 02 15:25:13.792931 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:13.792882    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls podName:aab15205-83f1-4672-93c8-61fb8a184d33 nodeName:}" failed. No retries permitted until 2026-11-02 15:25:14.292860281 +0000 UTC m=+1720.730762417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls") pod "maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" (UID: "aab15205-83f1-4672-93c8-61fb8a184d33") : secret "maas-api-e2e-shared-a-w5-6b2f61-serving-cert" not found
Nov 02 15:25:13.793007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.792974    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.794672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.794648    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:13.888407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:13.883027    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdw2\" (UniqueName: \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:14.307490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:14.307445    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:14.310134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:14.310096    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:14.525064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:14.525032    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:14.908922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:14.908885    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:25:14.910791 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:14.910764    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab15205_83f1_4672_93c8_61fb8a184d33.slice/crio-43463b5c97d7dfb354f9bf6d835145d741723b997f03a692925900f028bc5ee3 WatchSource:0}: Error finding container 43463b5c97d7dfb354f9bf6d835145d741723b997f03a692925900f028bc5ee3: Status 404 returned error can't find the container with id 43463b5c97d7dfb354f9bf6d835145d741723b997f03a692925900f028bc5ee3
Nov 02 15:25:15.524260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:15.524217    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" event={"ID":"aab15205-83f1-4672-93c8-61fb8a184d33","Type":"ContainerStarted","Data":"43463b5c97d7dfb354f9bf6d835145d741723b997f03a692925900f028bc5ee3"}
Nov 02 15:25:16.533239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:16.533180    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" event={"ID":"aab15205-83f1-4672-93c8-61fb8a184d33","Type":"ContainerStarted","Data":"e759eebb9a81da190e4338398f134565570d9315c8b368b497d1b2bae0427ac6"}
Nov 02 15:25:16.533727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:16.533290    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:16.676291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:16.676215    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" podStartSLOduration=2.831842475 podStartE2EDuration="3.676181548s" totalImagesPullingTime="844.339073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:16.673693592 +0000 UTC m=+1723.111595732" watchObservedRunningTime="2026-11-02 15:25:16.676181548 +0000 UTC m=+1723.114083688"
Nov 02 15:25:17.509563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:17.509528    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:25:18.623069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.623027    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:25:18.627476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.627453    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.679765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.679735    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-de9c4ba4-serving-cert\""
Nov 02 15:25:18.734676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.734640    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:25:18.783593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.783548    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.783777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.783678    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spr85\" (UniqueName: \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.783830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.783810    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.783949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.783932    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.889174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.888043    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.889174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.888144    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.889174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.888363    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.889174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.888430    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spr85\" (UniqueName: \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.892477 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.889639    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.897273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.897239    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.898703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.898669    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:18.954961 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:18.954928    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spr85\" (UniqueName: \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:19.241714 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:19.241681    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:19.623098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:19.623072    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:25:19.624598 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:19.624565    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66757923_f4f5_4cde_9dc8_a9b0f3f1dc11.slice/crio-db8249b2f021dba6fc0a690a555df483c60d30a2ddb25e4495c907f2578b14e8 WatchSource:0}: Error finding container db8249b2f021dba6fc0a690a555df483c60d30a2ddb25e4495c907f2578b14e8: Status 404 returned error can't find the container with id db8249b2f021dba6fc0a690a555df483c60d30a2ddb25e4495c907f2578b14e8
Nov 02 15:25:20.563593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:20.563565    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" event={"ID":"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11","Type":"ContainerStarted","Data":"db8249b2f021dba6fc0a690a555df483c60d30a2ddb25e4495c907f2578b14e8"}
Nov 02 15:25:21.571701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:21.571663    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" event={"ID":"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11","Type":"ContainerStarted","Data":"1d18cc607de4ffe7c1905059467b1aba52b598f1ce3ba461010de3d66776ee00"}
Nov 02 15:25:21.572126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:21.571752    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:21.702521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:21.702458    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" podStartSLOduration=2.792001332 podStartE2EDuration="3.702438569s" totalImagesPullingTime="910.437237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:21.699292519 +0000 UTC m=+1728.137194661" watchObservedRunningTime="2026-11-02 15:25:21.702438569 +0000 UTC m=+1728.140340713"
Nov 02 15:25:22.545862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:22.545828    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:25:27.584249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:27.584175    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:25:28.025445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.025411    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:25:28.029563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.029543    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.085138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.085107    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-eafc0d-openshift-default-dockercfg-xjz6b\""
Nov 02 15:25:28.137736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.137701    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:25:28.156947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.156912    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.156955    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157080    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvt9p\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157117    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157211    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157242    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157325    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157367    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.157571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.157511    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.261856    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.261972    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262006    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262071    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvt9p\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262103    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262143    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262170    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262238    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262288    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.262854    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.263870    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.264513    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.265277    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.267227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.265514    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.277260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.272717    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.279752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.279716    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.340967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.340898    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.353076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.353000    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvt9p\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\") pod \"e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") " pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:28.643957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:28.643876    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:29.087941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.087909    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:25:29.089302 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:29.089270    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ab13b7_1bf8_4a67_86fc_bdb411713174.slice/crio-499e16ef9410be49e20793c3ec3bc9b75a18f82f764bd7643afb5af325271640 WatchSource:0}: Error finding container 499e16ef9410be49e20793c3ec3bc9b75a18f82f764bd7643afb5af325271640: Status 404 returned error can't find the container with id 499e16ef9410be49e20793c3ec3bc9b75a18f82f764bd7643afb5af325271640
Nov 02 15:25:29.092063 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.092028    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:25:29.092180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.092114    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:25:29.092180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.092162    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:25:29.624914 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.624870    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerStarted","Data":"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"}
Nov 02 15:25:29.625114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.624930    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerStarted","Data":"499e16ef9410be49e20793c3ec3bc9b75a18f82f764bd7643afb5af325271640"}
Nov 02 15:25:29.643987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.643942    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:29.751487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:29.751425    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podStartSLOduration=2.7514105730000002 podStartE2EDuration="2.751410573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:29.746643656 +0000 UTC m=+1736.184545794" watchObservedRunningTime="2026-11-02 15:25:29.751410573 +0000 UTC m=+1736.189312799"
Nov 02 15:25:30.650331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:30.650302    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:31.639030 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:31.638996    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:31.640242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:31.640219    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:25:33.567253 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:33.566178    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:25:34.410605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:34.410562    4470 scope.go:122] "RemoveContainer" containerID="a9e84e41e9e3b11836cfbcd059a2ff50af58b4de6c6d369f2cd6112490ed7a7a"
Nov 02 15:25:34.427664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:34.427628    4470 scope.go:122] "RemoveContainer" containerID="bedfc0f48d6bcc7cc5dd6640a87ebe0c8e0051a3f31bd517bda2679e1268fde4"
Nov 02 15:25:36.313176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.313126    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:25:36.321539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.321513    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.333438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.333413    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.333579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.333552    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxg8j\" (UniqueName: \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.376675 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.376643    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:25:36.482908 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.482876    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-eafc0d-dockercfg-x9mwz\""
Nov 02 15:25:36.483204 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.483172    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-eafc0d\""
Nov 02 15:25:36.498215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.498165    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:25:36.518021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.517977    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.518531 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.518504    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxg8j\" (UniqueName: \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.524766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.520039    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.543354 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.540524    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.546149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.546112    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:25:36.606279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.602034    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxg8j\" (UniqueName: \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\") pod \"payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.620217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.619954    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j26d6\" (UniqueName: \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.620217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.620023    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.639359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.639328    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:36.728910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.723857    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j26d6\" (UniqueName: \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.728910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.723921    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.728910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.726151    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.786328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.786297    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j26d6\" (UniqueName: \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\") pod \"payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") " pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:36.931394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:36.931311    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:37.040064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.038931    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:25:37.043918 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:37.043868    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b518da0_159f_4c5d_a4f1_98f2f3db0900.slice/crio-43290182393ec2683734e2351df175ae5ecb0e074133146234d75051a5d9d0a8 WatchSource:0}: Error finding container 43290182393ec2683734e2351df175ae5ecb0e074133146234d75051a5d9d0a8: Status 404 returned error can't find the container with id 43290182393ec2683734e2351df175ae5ecb0e074133146234d75051a5d9d0a8
Nov 02 15:25:37.316473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.316448    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:25:37.318098 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:37.318066    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18ef7b1_7eb8_4bec_844e_f05421f90c90.slice/crio-0f0b38a367ad1ba5e60bf6086a4eeae29c7c5d83d466ebf6a594aeaa551f2bc2 WatchSource:0}: Error finding container 0f0b38a367ad1ba5e60bf6086a4eeae29c7c5d83d466ebf6a594aeaa551f2bc2: Status 404 returned error can't find the container with id 0f0b38a367ad1ba5e60bf6086a4eeae29c7c5d83d466ebf6a594aeaa551f2bc2
Nov 02 15:25:37.684004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.683965    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" event={"ID":"d18ef7b1-7eb8-4bec-844e-f05421f90c90","Type":"ContainerStarted","Data":"6a623d5fd941caa9526acfcd5b6cb8a048ddb8465fd375b75be49a63630154f4"}
Nov 02 15:25:37.684219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.684036    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" event={"ID":"d18ef7b1-7eb8-4bec-844e-f05421f90c90","Type":"ContainerStarted","Data":"0f0b38a367ad1ba5e60bf6086a4eeae29c7c5d83d466ebf6a594aeaa551f2bc2"}
Nov 02 15:25:37.684219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.684154    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:37.685642 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.685608    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" event={"ID":"9b518da0-159f-4c5d-a4f1-98f2f3db0900","Type":"ContainerStarted","Data":"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"}
Nov 02 15:25:37.685766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.685655    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:37.685766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.685675    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" event={"ID":"9b518da0-159f-4c5d-a4f1-98f2f3db0900","Type":"ContainerStarted","Data":"43290182393ec2683734e2351df175ae5ecb0e074133146234d75051a5d9d0a8"}
Nov 02 15:25:37.805042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:37.804989    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" podStartSLOduration=1.804973327 podStartE2EDuration="1.804973327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:37.804566047 +0000 UTC m=+1744.242468187" watchObservedRunningTime="2026-11-02 15:25:37.804973327 +0000 UTC m=+1744.242875468"
Nov 02 15:25:44.497347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.497285    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" podStartSLOduration=8.497264666 podStartE2EDuration="8.497264666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:37.977018673 +0000 UTC m=+1744.414920814" watchObservedRunningTime="2026-11-02 15:25:44.497264666 +0000 UTC m=+1750.935166810"
Nov 02 15:25:44.499220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.499184    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:25:44.505175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.505158    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.585886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.585841    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwdss\" (UniqueName: \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.586089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.585899    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.586089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.585989    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.586282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.586171    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.612380 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.612344    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-fd1339-serving-cert\""
Nov 02 15:25:44.615735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.615705    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:25:44.687222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.687142    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwdss\" (UniqueName: \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.687451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.687229    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.687451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.687278    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.687451 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:44.687394    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-serving-cert: secret "maas-api-e2e-shared-b-w1-fd1339-serving-cert" not found
Nov 02 15:25:44.687451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.687406    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.687645 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:44.687480    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls podName:79610468-a6cf-4732-9531-d4d430701010 nodeName:}" failed. No retries permitted until 2026-11-02 15:25:45.187455975 +0000 UTC m=+1751.625358093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls") pod "maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" (UID: "79610468-a6cf-4732-9531-d4d430701010") : secret "maas-api-e2e-shared-b-w1-fd1339-serving-cert" not found
Nov 02 15:25:44.688134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.688110    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.690206 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.690167    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:44.776473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:44.776319    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwdss\" (UniqueName: \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:45.210334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:45.210296    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:45.212828 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:45.212801    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:45.421388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:45.421350    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:45.820974 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:45.820941    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:25:45.822454 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:25:45.822424    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79610468_a6cf_4732_9531_d4d430701010.slice/crio-d43a4080390e554bdedaeb6b8732e44e9e09b4f15ec3e539dee88ea620a0f77a WatchSource:0}: Error finding container d43a4080390e554bdedaeb6b8732e44e9e09b4f15ec3e539dee88ea620a0f77a: Status 404 returned error can't find the container with id d43a4080390e554bdedaeb6b8732e44e9e09b4f15ec3e539dee88ea620a0f77a
Nov 02 15:25:46.781016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:46.780983    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" event={"ID":"79610468-a6cf-4732-9531-d4d430701010","Type":"ContainerStarted","Data":"d43a4080390e554bdedaeb6b8732e44e9e09b4f15ec3e539dee88ea620a0f77a"}
Nov 02 15:25:47.790013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:47.789958    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" event={"ID":"79610468-a6cf-4732-9531-d4d430701010","Type":"ContainerStarted","Data":"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"}
Nov 02 15:25:47.790458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:47.790065    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:47.939834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:47.939769    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" podStartSLOduration=3.063368528 podStartE2EDuration="3.93974894s" totalImagesPullingTime="876.380412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:25:47.93751753 +0000 UTC m=+1754.375419673" watchObservedRunningTime="2026-11-02 15:25:47.93974894 +0000 UTC m=+1754.377651083"
Nov 02 15:25:48.696761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:48.696733    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:25:48.696947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:48.696814    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:25:53.801921 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:53.801893    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:25:59.556423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.556390    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:25:59.560831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.560811    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.616073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.616012    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-4c5b4e-serving-cert\""
Nov 02 15:25:59.669179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.669145    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:25:59.701723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.701680    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.701946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.701780    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bms52\" (UniqueName: \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.701946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.701830    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.701946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.701857    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.813605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.813499    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.814433 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.814401    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bms52\" (UniqueName: \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.814658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.814475    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.814658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.814506    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.816291 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:59.814264    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-serving-cert: secret "maas-api-e2e-shared-b-w3-4c5b4e-serving-cert" not found
Nov 02 15:25:59.816291 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:25:59.815401    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls podName:416fb9f2-d1f2-443d-a079-90df3a9a52e3 nodeName:}" failed. No retries permitted until 2026-11-02 15:26:00.315340295 +0000 UTC m=+1766.753242432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls") pod "maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" (UID: "416fb9f2-d1f2-443d-a079-90df3a9a52e3") : secret "maas-api-e2e-shared-b-w3-4c5b4e-serving-cert" not found
Nov 02 15:25:59.824057 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.816950    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.847218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.846246    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:25:59.881627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:25:59.881592    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bms52\" (UniqueName: \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:00.326219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:00.326151    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:00.331890 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:00.331859    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:00.475517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:00.475473    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:00.867906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:00.867872    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:26:00.869069 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:00.869037    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416fb9f2_d1f2_443d_a079_90df3a9a52e3.slice/crio-a7c0c7586cc1a1d100720fb9a5872db990d73cd8715578a4a31a781f06531262 WatchSource:0}: Error finding container a7c0c7586cc1a1d100720fb9a5872db990d73cd8715578a4a31a781f06531262: Status 404 returned error can't find the container with id a7c0c7586cc1a1d100720fb9a5872db990d73cd8715578a4a31a781f06531262
Nov 02 15:26:00.880523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:00.880496    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" event={"ID":"416fb9f2-d1f2-443d-a079-90df3a9a52e3","Type":"ContainerStarted","Data":"a7c0c7586cc1a1d100720fb9a5872db990d73cd8715578a4a31a781f06531262"}
Nov 02 15:26:01.889104 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:01.889053    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" event={"ID":"416fb9f2-d1f2-443d-a079-90df3a9a52e3","Type":"ContainerStarted","Data":"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"}
Nov 02 15:26:01.889603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:01.889132    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:02.027514 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:02.027394    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" podStartSLOduration=2.187470323 podStartE2EDuration="3.027378079s" totalImagesPullingTime="839.907756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:25:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:02.022921566 +0000 UTC m=+1768.460823710" watchObservedRunningTime="2026-11-02 15:26:02.027378079 +0000 UTC m=+1768.465280219"
Nov 02 15:26:03.566958 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:26:03.566918    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:26:07.689382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.689341    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:26:07.694608 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.694455    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757119    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757308    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757338    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfm9w\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757367    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757414    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757469    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757747    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757854    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.758572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.757922    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.775347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.775312    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:26:07.808905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.808370    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-53fd96-openshift-default-dockercfg-7sjxw\""
Nov 02 15:26:07.859094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859048    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859088    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859245    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859311    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859334    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfm9w\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859350    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859464    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859529    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859576    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859673    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859576    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.859728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.859716    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.860091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.860069    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.860201 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.860129    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.861697 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.861672    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.861967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.861949    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.901879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.901851    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:26:07.966261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.947091    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:07.966555 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:07.951132    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfm9w\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\") pod \"e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") " pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:08.012505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:08.012465    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:08.502661 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:08.502636    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:26:08.504267 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:08.504239    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12f8cb24_efd6_4fbc_b353_86bb64d2c71a.slice/crio-68815de23e37112d84dc8abca7173000ed6f4ef49cec78dea9ccc46a24f21372 WatchSource:0}: Error finding container 68815de23e37112d84dc8abca7173000ed6f4ef49cec78dea9ccc46a24f21372: Status 404 returned error can't find the container with id 68815de23e37112d84dc8abca7173000ed6f4ef49cec78dea9ccc46a24f21372
Nov 02 15:26:08.507022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:08.506981    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:08.507113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:08.507057    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:08.507113 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:08.507089    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:09.007427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:09.007392    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerStarted","Data":"3143030bf4f3ac03d02943ea1dc7fde09ca407b8193d61a3011fe9e3e7ac5612"}
Nov 02 15:26:09.007871 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:09.007445    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerStarted","Data":"68815de23e37112d84dc8abca7173000ed6f4ef49cec78dea9ccc46a24f21372"}
Nov 02 15:26:09.013794 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:09.013767    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:09.020335 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:09.020311    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:09.142883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:09.142816    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podStartSLOduration=2.142800845 podStartE2EDuration="2.142800845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:09.138485329 +0000 UTC m=+1775.576387471" watchObservedRunningTime="2026-11-02 15:26:09.142800845 +0000 UTC m=+1775.580702986"
Nov 02 15:26:10.023218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:10.021554    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:10.023218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:10.022961    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:26:15.622212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.620415    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:26:15.629218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.627791    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:15.674663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.674624    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:26:15.762883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.735742    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-53fd96-dockercfg-v94nw\""
Nov 02 15:26:15.764496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.763331    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-53fd96\""
Nov 02 15:26:15.805376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.805334    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:26:15.817566 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.817531    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbbt\" (UniqueName: \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:15.818511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.817983    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:15.883708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.883662    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:15.907506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.907462    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:26:15.985228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.985169    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbbt\" (UniqueName: \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:15.985417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.985313    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:15.985417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.985340    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:15.985417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.985400    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxtgk\" (UniqueName: \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:15.986112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:15.986088    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:16.046732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.046693    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbbt\" (UniqueName: \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\") pod \"payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:16.086759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.086720    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:16.086920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.086803    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxtgk\" (UniqueName: \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:16.087569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.087547    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:16.148495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.148409    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxtgk\" (UniqueName: \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\") pod \"payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:16.199025 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.198988    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:16.266996 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.266966    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:16.586172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.586086    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:26:16.588605 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:16.588574    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78b2172_9204_4c57_89e4_b390cd1a14ef.slice/crio-f3e0cbb968777ca3f03bc904c475c13c518a33fe4abd02c3244fc0ab55aab845 WatchSource:0}: Error finding container f3e0cbb968777ca3f03bc904c475c13c518a33fe4abd02c3244fc0ab55aab845: Status 404 returned error can't find the container with id f3e0cbb968777ca3f03bc904c475c13c518a33fe4abd02c3244fc0ab55aab845
Nov 02 15:26:16.701300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:16.701270    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:26:16.703597 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:16.703567    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b4f1f0_baa2_4f05_84af_f9ec411464a2.slice/crio-19321231173f3b114ff52b678e1c6e2fec21a32ab57da604bc85a57a2a5428a8 WatchSource:0}: Error finding container 19321231173f3b114ff52b678e1c6e2fec21a32ab57da604bc85a57a2a5428a8: Status 404 returned error can't find the container with id 19321231173f3b114ff52b678e1c6e2fec21a32ab57da604bc85a57a2a5428a8
Nov 02 15:26:17.069852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.069809    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" event={"ID":"d4b4f1f0-baa2-4f05-84af-f9ec411464a2","Type":"ContainerStarted","Data":"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"}
Nov 02 15:26:17.070041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.069884    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" event={"ID":"d4b4f1f0-baa2-4f05-84af-f9ec411464a2","Type":"ContainerStarted","Data":"19321231173f3b114ff52b678e1c6e2fec21a32ab57da604bc85a57a2a5428a8"}
Nov 02 15:26:17.070041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.069916    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:17.071884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.071853    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" event={"ID":"d78b2172-9204-4c57-89e4-b390cd1a14ef","Type":"ContainerStarted","Data":"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"}
Nov 02 15:26:17.072009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.071905    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" event={"ID":"d78b2172-9204-4c57-89e4-b390cd1a14ef","Type":"ContainerStarted","Data":"f3e0cbb968777ca3f03bc904c475c13c518a33fe4abd02c3244fc0ab55aab845"}
Nov 02 15:26:17.072009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.071951    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:17.189988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.189931    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" podStartSLOduration=2.189917153 podStartE2EDuration="2.189917153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:17.186252647 +0000 UTC m=+1783.624154789" watchObservedRunningTime="2026-11-02 15:26:17.189917153 +0000 UTC m=+1783.627819333"
Nov 02 15:26:17.307246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:17.307163    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" podStartSLOduration=2.307147004 podStartE2EDuration="2.307147004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:17.305580493 +0000 UTC m=+1783.743482635" watchObservedRunningTime="2026-11-02 15:26:17.307147004 +0000 UTC m=+1783.745049144"
Nov 02 15:26:28.082045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:28.082012    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:26:28.082521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:28.082116    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:26:33.567511 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:26:33.567467    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:26:33.963547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:33.963520    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:26:33.966886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:33.966857    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:26:39.291092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.291046    4470 generic.go:356] "Generic (PLEG): container finished" podID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerID="88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5" exitCode=0
Nov 02 15:26:39.291708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.291128    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerDied","Data":"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"}
Nov 02 15:26:39.291788 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.291734    4470 scope.go:122] "RemoveContainer" containerID="88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"
Nov 02 15:26:39.293243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293181    4470 generic.go:356] "Generic (PLEG): container finished" podID="33eaeb67-0da1-4614-b321-520f16cb8607" containerID="9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e" exitCode=0
Nov 02 15:26:39.293361 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293261    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerDied","Data":"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"}
Nov 02 15:26:39.293671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293644    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:39.293716 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293690    4470 scope.go:122] "RemoveContainer" containerID="9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"
Nov 02 15:26:39.293758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293711    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:39.293758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.293751    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:39.295348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.295316    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:39.295460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.295397    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:39.295460 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:39.295452    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:40.312230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:40.312151    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerStarted","Data":"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"}
Nov 02 15:26:40.314731 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:40.314694    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerStarted","Data":"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"}
Nov 02 15:26:41.138585 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.138542    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:26:41.139378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.139347    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:41.139520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.139400    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:41.225409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.225375    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:26:41.229548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.229520    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.230816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.230793    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:26:41.232293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.232266    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:41.232385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.232320    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332440    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332567    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332621    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332689    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332731    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z7xh\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332848    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332916    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.332978    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.333222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.333011    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.340221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.338273    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:26:41.374215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.369173    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-d6b9c6-openshift-default-dockercfg-7bv76\""
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443349    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443431    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443526    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443574    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z7xh\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443709    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443778    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443826    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443856    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.444219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.443942    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.527222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.523338    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.527222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.523342    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.527222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.524641    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.527222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.525555    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.527222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.525916    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.733261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.732990    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:41.766259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:41.738448    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:42.090315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.086740    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:26:42.102770 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.102670    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z7xh\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:42.104474 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.104448    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.129658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.104340    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:26:42.132928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.132745    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\") pod \"e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") " pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:42.139297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.139263    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:42.139559 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.139532    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:42.145289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.145262    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:42.161088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.160180    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-885c69-openshift-default-dockercfg-wzvl4\""
Nov 02 15:26:42.248079 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248037    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:42.248321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248137    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:42.248546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248519    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248561    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248612    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248672    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9drf\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248731    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248781    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.248973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.248954    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.249037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.249011    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.249128 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.249111    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.351598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351399    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.351598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351463    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351620    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351661    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351738    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351847    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351875    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351921    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.352002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.351972    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9drf\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.355620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.355575    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.357126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.356658    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.359118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.359086    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.359539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.359514    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.359906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.359886    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.361509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.361463    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.370359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.370291    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.464906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.464567    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.479542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.479445    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9drf\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\") pod \"e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") " pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.514044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.512940    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:42.974307 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:42.969719    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:26:43.122035 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:43.120887    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae98f0a5_0b8b_454c_8f16_d4a0f949277a.slice/crio-7fcaaf1c26df9cda91c2bfe55401802513aef89c9c1d1ce90b1251f18dd4f4d4 WatchSource:0}: Error finding container 7fcaaf1c26df9cda91c2bfe55401802513aef89c9c1d1ce90b1251f18dd4f4d4: Status 404 returned error can't find the container with id 7fcaaf1c26df9cda91c2bfe55401802513aef89c9c1d1ce90b1251f18dd4f4d4
Nov 02 15:26:43.138448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.138037    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:43.138448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.138137    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:43.138801 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.138765    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.138907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.138843    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.138907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.138892    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.231202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.230670    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:43.231202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.230746    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:43.231402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.231311    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:26:43.231780 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:43.231749    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6916f781_7088_43a3_811c_ab761ae73391.slice/crio-3f59860629ce1288ca4b2d62c80d349be425d11b7bb718fedd0efbf4f0e445ab WatchSource:0}: Error finding container 3f59860629ce1288ca4b2d62c80d349be425d11b7bb718fedd0efbf4f0e445ab: Status 404 returned error can't find the container with id 3f59860629ce1288ca4b2d62c80d349be425d11b7bb718fedd0efbf4f0e445ab
Nov 02 15:26:43.235305 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.235257    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.235394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.235354    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.235432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.235396    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:26:43.250617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.250591    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:26:43.477275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.476134    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerStarted","Data":"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"}
Nov 02 15:26:43.477275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.476228    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerStarted","Data":"7fcaaf1c26df9cda91c2bfe55401802513aef89c9c1d1ce90b1251f18dd4f4d4"}
Nov 02 15:26:43.515142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.494556    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerStarted","Data":"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"}
Nov 02 15:26:43.515142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.494631    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerStarted","Data":"3f59860629ce1288ca4b2d62c80d349be425d11b7bb718fedd0efbf4f0e445ab"}
Nov 02 15:26:43.516815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.516734    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:43.809264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.806169    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podStartSLOduration=2.806148602 podStartE2EDuration="2.806148602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:43.769795556 +0000 UTC m=+1810.207697768" watchObservedRunningTime="2026-11-02 15:26:43.806148602 +0000 UTC m=+1810.244050745"
Nov 02 15:26:43.837712 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:43.837639    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podStartSLOduration=2.83761703 podStartE2EDuration="2.83761703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:43.629838772 +0000 UTC m=+1810.067740914" watchObservedRunningTime="2026-11-02 15:26:43.83761703 +0000 UTC m=+1810.275519174"
Nov 02 15:26:44.145270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.140001    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:44.145270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.140071    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:44.154214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.151338    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:44.154214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.151538    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:26:44.154214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.151609    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:26:44.248546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.248512    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:44.248843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.248802    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:44.528842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.528804    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:44.533579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.533550    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:44.534916 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:44.534895    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:26:45.138647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.138602    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:45.138844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.138672    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:45.151405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.151377    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:45.231303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.231269    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:45.231565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.231536    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:45.540742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.540709    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:45.541680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:45.541661    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:26:46.049646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:46.049609    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:26:46.138708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:46.138662    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused" start-of-body=
Nov 02 15:26:46.138898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:46.138747    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.103:15021/healthz/ready\": dial tcp 10.133.0.103:15021: connect: connection refused"
Nov 02 15:26:46.232211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:46.231650    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused" start-of-body=
Nov 02 15:26:46.232211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:46.231715    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.102:15021/healthz/ready\": dial tcp 10.133.0.102:15021: connect: connection refused"
Nov 02 15:26:47.144224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:47.144159    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:26:47.145385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:47.145361    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:26:47.235388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:47.235356    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:26:47.236527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:47.236504    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:26:48.335430 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.335394    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:26:48.339329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.339306    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.428737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.428702    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de9c4ba4\"/\"default-dockercfg-2p2hv\""
Nov 02 15:26:48.435066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.435014    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:26:48.444500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.444460    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-de9c4ba4\"/\"e2e-lifecycle-model-de9c4ba4-kserve-self-signed-certs\""
Nov 02 15:26:48.444500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.444482    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de9c4ba4\"/\"openshift-service-ca.crt\""
Nov 02 15:26:48.444675 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.444552    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-de9c4ba4\"/\"kube-root-ca.crt\""
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.508845    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.508909    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.508983    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95x7t\" (UniqueName: \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.509058    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.509088    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.509458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.509265    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611292    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611451    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611496    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611572    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95x7t\" (UniqueName: \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611642    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.611676    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.612334    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.612576    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.612751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.612597    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.616861 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.616831    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.617459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.617435    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.749214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.746879    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95x7t\" (UniqueName: \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\") pod \"e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") " pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:48.953447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:48.953408    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:49.343349 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:49.343314    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:26:49.344968 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:49.344929    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode971f64c_b234_42b7_aba2_99ffb6e68532.slice/crio-108cbc572abaf193817a772cdb8f79e823283ecc8aa2db91e05d336ca0fced03 WatchSource:0}: Error finding container 108cbc572abaf193817a772cdb8f79e823283ecc8aa2db91e05d336ca0fced03: Status 404 returned error can't find the container with id 108cbc572abaf193817a772cdb8f79e823283ecc8aa2db91e05d336ca0fced03
Nov 02 15:26:49.587848 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:49.587806    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" event={"ID":"e971f64c-b234-42b7-aba2-99ffb6e68532","Type":"ContainerStarted","Data":"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"}
Nov 02 15:26:49.587848 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:49.587857    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" event={"ID":"e971f64c-b234-42b7-aba2-99ffb6e68532","Type":"ContainerStarted","Data":"108cbc572abaf193817a772cdb8f79e823283ecc8aa2db91e05d336ca0fced03"}
Nov 02 15:26:49.713898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:49.713841    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" podStartSLOduration=1.7138262370000001 podStartE2EDuration="1.713826237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:49.709630533 +0000 UTC m=+1816.147532675" watchObservedRunningTime="2026-11-02 15:26:49.713826237 +0000 UTC m=+1816.151728377"
Nov 02 15:26:49.954290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:49.954255    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:52.487214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.483150    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:26:52.491216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.490148    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.504210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.502978    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.504210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.503203    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gx6l\" (UniqueName: \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.550995 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.550955    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:26:52.608125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.606516    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-d6b9c6\""
Nov 02 15:26:52.608125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.606700    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-d6b9c6-dockercfg-2rwbm\""
Nov 02 15:26:52.629230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.624239    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.629230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.624578    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gx6l\" (UniqueName: \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.629230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.625970    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.697248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.695480    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:26:52.721223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.719242    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:52.755607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.755574    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:26:52.830458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.830426    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gx6l\" (UniqueName: \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\") pod \"payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.832543 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.832511    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:52.835435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.834784    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:52.835435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.835220    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk7nl\" (UniqueName: \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:52.941488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.941445    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:52.941721 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.941698    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7nl\" (UniqueName: \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:52.943219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:52.943176    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:53.015607 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.015526    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk7nl\" (UniqueName: \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\") pod \"payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:53.056957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.056919    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:53.237374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.237349    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:26:53.238653 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:53.238622    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b23d5ba_d054_4835_8f5a_e6886e76c1b3.slice/crio-a3b9f200c686b107e6d485e5dfd2c27596501772d9c129aeab4b912dbb7f4b95 WatchSource:0}: Error finding container a3b9f200c686b107e6d485e5dfd2c27596501772d9c129aeab4b912dbb7f4b95: Status 404 returned error can't find the container with id a3b9f200c686b107e6d485e5dfd2c27596501772d9c129aeab4b912dbb7f4b95
Nov 02 15:26:53.444590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.444558    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:26:53.445383 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:53.445354    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c27b68_8549_42d5_841d_3c396bcaa889.slice/crio-19126366b123e6867c3e8c5e3c9a789f3a140fd754b9f5d8a661dcb858e8679f WatchSource:0}: Error finding container 19126366b123e6867c3e8c5e3c9a789f3a140fd754b9f5d8a661dcb858e8679f: Status 404 returned error can't find the container with id 19126366b123e6867c3e8c5e3c9a789f3a140fd754b9f5d8a661dcb858e8679f
Nov 02 15:26:53.644279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.644180    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:26:53.649005 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.648981    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.653965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.653935    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" event={"ID":"28c27b68-8549-42d5-841d-3c396bcaa889","Type":"ContainerStarted","Data":"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"}
Nov 02 15:26:53.654135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.653979    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" event={"ID":"28c27b68-8549-42d5-841d-3c396bcaa889","Type":"ContainerStarted","Data":"19126366b123e6867c3e8c5e3c9a789f3a140fd754b9f5d8a661dcb858e8679f"}
Nov 02 15:26:53.654135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.654001    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:26:53.655821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.655792    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" event={"ID":"5b23d5ba-d054-4835-8f5a-e6886e76c1b3","Type":"ContainerStarted","Data":"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"}
Nov 02 15:26:53.655949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.655845    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" event={"ID":"5b23d5ba-d054-4835-8f5a-e6886e76c1b3","Type":"ContainerStarted","Data":"a3b9f200c686b107e6d485e5dfd2c27596501772d9c129aeab4b912dbb7f4b95"}
Nov 02 15:26:53.655949 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.655881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:26:53.703654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.703623    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-eafc0d-serving-cert\""
Nov 02 15:26:53.710422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.710389    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:26:53.752897 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.752865    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.753182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.753145    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.753366 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.753301    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.753533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.753514    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkhrw\" (UniqueName: \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.855628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.855592    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.855797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.855705    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.855797 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:26:53.855755    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-serving-cert: secret "maas-api-e2e-api-a-eafc0d-serving-cert" not found
Nov 02 15:26:53.855917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.855803    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.855917 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:26:53.855824    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls podName:79e443ad-b25f-4f75-9d0e-0ac310e712da nodeName:}" failed. No retries permitted until 2026-11-02 15:26:54.355801361 +0000 UTC m=+1820.793703503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls") pod "maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" (UID: "79e443ad-b25f-4f75-9d0e-0ac310e712da") : secret "maas-api-e2e-api-a-eafc0d-serving-cert" not found
Nov 02 15:26:53.855917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.855879    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkhrw\" (UniqueName: \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.856519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.856488    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.858673 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.858649    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.917361 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.917287    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkhrw\" (UniqueName: \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:53.947362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:53.947292    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" podStartSLOduration=1.94727509 podStartE2EDuration="1.94727509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:53.945653756 +0000 UTC m=+1820.383555896" watchObservedRunningTime="2026-11-02 15:26:53.94727509 +0000 UTC m=+1820.385177231"
Nov 02 15:26:54.066175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:54.066117    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" podStartSLOduration=2.066100373 podStartE2EDuration="2.066100373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:54.065312391 +0000 UTC m=+1820.503214533" watchObservedRunningTime="2026-11-02 15:26:54.066100373 +0000 UTC m=+1820.504002515"
Nov 02 15:26:54.392889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:54.392853    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:54.395345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:54.395317    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") pod \"maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:54.564726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:54.564694    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:54.956896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:54.952269    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:26:54.956896 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:26:54.954513    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e443ad_b25f_4f75_9d0e_0ac310e712da.slice/crio-78ab1d3561223e3ca9cb6663c3b232220234d4bd028b2f526d37fafbb2398565 WatchSource:0}: Error finding container 78ab1d3561223e3ca9cb6663c3b232220234d4bd028b2f526d37fafbb2398565: Status 404 returned error can't find the container with id 78ab1d3561223e3ca9cb6663c3b232220234d4bd028b2f526d37fafbb2398565
Nov 02 15:26:55.672803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:55.672769    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" event={"ID":"79e443ad-b25f-4f75-9d0e-0ac310e712da","Type":"ContainerStarted","Data":"78ab1d3561223e3ca9cb6663c3b232220234d4bd028b2f526d37fafbb2398565"}
Nov 02 15:26:56.681923 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:56.681887    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" event={"ID":"79e443ad-b25f-4f75-9d0e-0ac310e712da","Type":"ContainerStarted","Data":"2f39e6b0abeb345be623900418ee009a415172b0ba35c5ec73075643bbe9ae0f"}
Nov 02 15:26:56.682363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:56.681975    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:26:56.800639 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:56.800507    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" podStartSLOduration=3.022408361 podStartE2EDuration="3.800488715s" totalImagesPullingTime="778.080354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:26:56.799537557 +0000 UTC m=+1823.237439752" watchObservedRunningTime="2026-11-02 15:26:56.800488715 +0000 UTC m=+1823.238390857"
Nov 02 15:26:58.954670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:58.954628    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:58.967312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:58.967284    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:26:59.727915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:26:59.727877    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:27:02.695734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:02.695703    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:27:03.567271 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:27:03.567234    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:27:04.666864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:04.666831    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:27:04.667348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:04.666911    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:27:06.426624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.426582    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:27:06.427276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.426850    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery" containerID="cri-o://c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059" gracePeriod=30
Nov 02 15:27:06.739864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.739835    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:27:06.762061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.762024    4470 generic.go:356] "Generic (PLEG): container finished" podID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerID="c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059" exitCode=0
Nov 02 15:27:06.762281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.762071    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"
Nov 02 15:27:06.762281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.762086    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" event={"ID":"8fae4b29-d226-4bc9-971b-bdddd302f1ca","Type":"ContainerDied","Data":"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"}
Nov 02 15:27:06.762281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.762149    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj" event={"ID":"8fae4b29-d226-4bc9-971b-bdddd302f1ca","Type":"ContainerDied","Data":"8fe8bc791a75521b86642811f5229127547aa33f52df000d527e9e113dd97d48"}
Nov 02 15:27:06.762281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.762177    4470 scope.go:122] "RemoveContainer" containerID="c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"
Nov 02 15:27:06.772690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.772666    4470 scope.go:122] "RemoveContainer" containerID="c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"
Nov 02 15:27:06.773017 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:27:06.772997    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059\": container with ID starting with c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059 not found: ID does not exist" containerID="c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"
Nov 02 15:27:06.773097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.773033    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059"} err="failed to get container status \"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059\": rpc error: code = NotFound desc = could not find container \"c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059\": container with ID starting with c45d86cd07d8de0db5c081bde9a94effdbcdac136efb39f365d1d5954633b059 not found: ID does not exist"
Nov 02 15:27:06.911161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.911124    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.911447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.911279    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.911810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.911783    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.912232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.912166    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.912355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.912252    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.912672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.912650    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.912786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.912720    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\") pod \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\" (UID: \"8fae4b29-d226-4bc9-971b-bdddd302f1ca\") "
Nov 02 15:27:06.913765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.913717    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:27:06.914003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.913976    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:27:06.914102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.914026    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:27:06.915650 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.915624    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:27:06.916094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.916075    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:27:06.919648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.919619    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "kube-api-access-9sbwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:27:06.920872 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:06.920845    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs" pod "8fae4b29-d226-4bc9-971b-bdddd302f1ca" (UID: "8fae4b29-d226-4bc9-971b-bdddd302f1ca"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:27:07.015832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015736    4470 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fae4b29-d226-4bc9-971b-bdddd302f1ca-local-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.015832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015769    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.015832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015783    4470 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-kubeconfig\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.015832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015796    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-ca-configmap\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.015832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015811    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-istio-csr-dns-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.016151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015868    4470 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fae4b29-d226-4bc9-971b-bdddd302f1ca-cacerts\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.016151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.015883    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sbwt\" (UniqueName: \"kubernetes.io/projected/8fae4b29-d226-4bc9-971b-bdddd302f1ca-kube-api-access-9sbwt\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:27:07.207240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.207204    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:27:07.214422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.214387    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d6dj"]
Nov 02 15:27:07.708496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:07.708456    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" path="/var/lib/kubelet/pods/8fae4b29-d226-4bc9-971b-bdddd302f1ca/volumes"
Nov 02 15:27:21.854181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.854082    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:27:21.854980 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.854956    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery"
Nov 02 15:27:21.855146 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.855125    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery"
Nov 02 15:27:21.855392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.855377    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fae4b29-d226-4bc9-971b-bdddd302f1ca" containerName="discovery"
Nov 02 15:27:21.861148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.861118    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:21.924240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.921509    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-303ba8-openshift-default-dockercfg-zf496\""
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963580    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963715    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963745    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963817    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963878    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963905    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.963947    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.964044    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhwhj\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:21.964116    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.042125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.001269    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.066886    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.067079    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.067258    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.067283    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.068632    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.082250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.069351    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.069601    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.067373    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.162385    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.162421    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.162474    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.162585    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwhj\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.142573    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.167074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.163521    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.285718 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.283356    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.285718 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.283688    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.292477 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.286571    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.295328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.295300    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhwhj\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\") pod \"e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") " pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.503451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.503408    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:22.952240 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.948269    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:27:22.956365 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.956325    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:22.956505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.956407    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:22.956647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:22.956438    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:23.906696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:23.906650    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerStarted","Data":"8c637e38f2ab3d17a6b9b3a3fb8b45674970c5b50aacee87c83731503a812573"}
Nov 02 15:27:23.906919 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:23.906725    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerStarted","Data":"b6d82a01efe2b104e79b08e784a74157233a5bc6454a2b6c5088f18e06e2cf54"}
Nov 02 15:27:24.128853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:24.128789    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podStartSLOduration=3.128772231 podStartE2EDuration="3.128772231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:24.076545079 +0000 UTC m=+1850.514447290" watchObservedRunningTime="2026-11-02 15:27:24.128772231 +0000 UTC m=+1850.566674435"
Nov 02 15:27:24.504336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:24.504292    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:24.510926 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:24.510898    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:24.913683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:24.913592    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:24.914598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:24.914576    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:27:30.823371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.822242    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:27:30.837041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.837012    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:30.860815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.860783    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:27:30.887637 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.887590    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djm9j\" (UniqueName: \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:30.887866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.887828    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:30.906256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.906227    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-303ba8\""
Nov 02 15:27:30.906544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:30.906521    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-303ba8-dockercfg-qjnps\""
Nov 02 15:27:31.031221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.030928    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djm9j\" (UniqueName: \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:31.031221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.031110    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:31.057126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.057077    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:31.121442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.121130    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:27:31.129738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.129705    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djm9j\" (UniqueName: \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\") pod \"payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:31.148234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.144629    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.148234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.146454    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:27:31.161025 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.160997    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:31.221994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.221924    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.222256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.222232    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d989t\" (UniqueName: \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.323797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.323756    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d989t\" (UniqueName: \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.323968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.323919    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.324707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.324686    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.385525 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.385458    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d989t\" (UniqueName: \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\") pod \"payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") " pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.467117 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.467071    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:31.555063 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:27:31.555005    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9400a595_d018_4feb_a913_9d2ec496cbf7.slice/crio-a645d0552a14f15d15716474686bfe7b4999adc27ab3b295e93e4432671d7222 WatchSource:0}: Error finding container a645d0552a14f15d15716474686bfe7b4999adc27ab3b295e93e4432671d7222: Status 404 returned error can't find the container with id a645d0552a14f15d15716474686bfe7b4999adc27ab3b295e93e4432671d7222
Nov 02 15:27:31.563065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.563038    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:27:31.858149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:31.858124    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:27:31.859296 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:27:31.859271    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248ccda4_84a5_4a23_9fb2_2f782e0e34c6.slice/crio-96ec5e11ce024c8f20f173718c80dd33a903752529f23ca879f939c01ff55357 WatchSource:0}: Error finding container 96ec5e11ce024c8f20f173718c80dd33a903752529f23ca879f939c01ff55357: Status 404 returned error can't find the container with id 96ec5e11ce024c8f20f173718c80dd33a903752529f23ca879f939c01ff55357
Nov 02 15:27:32.137823 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.137786    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" event={"ID":"9400a595-d018-4feb-a913-9d2ec496cbf7","Type":"ContainerStarted","Data":"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"}
Nov 02 15:27:32.138014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.137854    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" event={"ID":"9400a595-d018-4feb-a913-9d2ec496cbf7","Type":"ContainerStarted","Data":"a645d0552a14f15d15716474686bfe7b4999adc27ab3b295e93e4432671d7222"}
Nov 02 15:27:32.138014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.137883    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:32.139662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.139632    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" event={"ID":"248ccda4-84a5-4a23-9fb2-2f782e0e34c6","Type":"ContainerStarted","Data":"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"}
Nov 02 15:27:32.139816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.139689    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:32.139816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.139713    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" event={"ID":"248ccda4-84a5-4a23-9fb2-2f782e0e34c6","Type":"ContainerStarted","Data":"96ec5e11ce024c8f20f173718c80dd33a903752529f23ca879f939c01ff55357"}
Nov 02 15:27:32.254276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.254223    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" podStartSLOduration=2.254205575 podStartE2EDuration="2.254205575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:32.253766272 +0000 UTC m=+1858.691668448" watchObservedRunningTime="2026-11-02 15:27:32.254205575 +0000 UTC m=+1858.692107719"
Nov 02 15:27:32.393223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:32.392286    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" podStartSLOduration=2.392170128 podStartE2EDuration="2.392170128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:32.375488623 +0000 UTC m=+1858.813390765" watchObservedRunningTime="2026-11-02 15:27:32.392170128 +0000 UTC m=+1858.830072269"
Nov 02 15:27:33.566705 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:27:33.566651    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:27:36.072929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.072895    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:27:36.078889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.078858    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126110    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnf7r\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126205    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126252    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126334    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126391    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126420    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126487    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126539    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.126746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.126610    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.144228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.139319    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-468af4-openshift-default-dockercfg-mfjmt\""
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254394    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254478    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254555    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254660    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnf7r\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254725    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254777    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254881    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254954    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.254984    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.282056    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.282637    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.282748    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.290454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.283418    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.400110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.300543    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.400110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.365108    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.403019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.400866    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnf7r\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.406225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.406146    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.467386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.463544    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\") pod \"e2e-inf-a-468af4-openshift-default-dd578d549-dbppz\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") " pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:36.495222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.488429    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:27:36.694367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:36.694328    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:37.141519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.141492    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:27:37.143422 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:27:37.143385    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c7016c_1dc1_4269_9a32_ed3c1c264e1a.slice/crio-f95b525d170bc79deccdfa00c7f4a5944c61ed951ac15bdd1cd134617e26a322 WatchSource:0}: Error finding container f95b525d170bc79deccdfa00c7f4a5944c61ed951ac15bdd1cd134617e26a322: Status 404 returned error can't find the container with id f95b525d170bc79deccdfa00c7f4a5944c61ed951ac15bdd1cd134617e26a322
Nov 02 15:27:37.146565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.146528    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:37.146651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.146604    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:37.146651 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.146633    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:37.422501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.422406    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerStarted","Data":"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"}
Nov 02 15:27:37.422501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.422465    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerStarted","Data":"f95b525d170bc79deccdfa00c7f4a5944c61ed951ac15bdd1cd134617e26a322"}
Nov 02 15:27:37.550514 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.550456    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podStartSLOduration=1.5504382749999999 podStartE2EDuration="1.550438275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:37.548690216 +0000 UTC m=+1863.986592392" watchObservedRunningTime="2026-11-02 15:27:37.550438275 +0000 UTC m=+1863.988340434"
Nov 02 15:27:37.706208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:37.699287    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:38.700557 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:38.700523    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:39.437659 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:39.437622    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:39.438601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:39.438575    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:27:40.669908 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.669869    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:27:40.674903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.674882    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.779106 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.779070    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-53fd96-serving-cert\""
Nov 02 15:27:40.783786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.783762    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:27:40.791296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.791264    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.791480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.791429    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.791552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.791497    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.791864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.791833    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5xxw\" (UniqueName: \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.893550 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.893509    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.893768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.893624    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.893768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.893671    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.893892 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.893772    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5xxw\" (UniqueName: \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.893892 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:27:40.893820    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-serving-cert: secret "maas-api-e2e-api-a-53fd96-serving-cert" not found
Nov 02 15:27:40.893999 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:27:40.893904    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls podName:1a0cde38-b591-4c62-b753-1ccf1d17dd90 nodeName:}" failed. No retries permitted until 2026-11-02 15:27:41.393879583 +0000 UTC m=+1867.831781715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls") pod "maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" (UID: "1a0cde38-b591-4c62-b753-1ccf1d17dd90") : secret "maas-api-e2e-api-a-53fd96-serving-cert" not found
Nov 02 15:27:40.894424 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.894397    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.896274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.896251    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:40.967954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:40.967916    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5xxw\" (UniqueName: \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:41.405539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:41.405450    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:41.408046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:41.408016    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") pod \"maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:41.593611 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:41.593578    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:41.984440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:41.983357    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:27:42.460941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:42.460897    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" event={"ID":"1a0cde38-b591-4c62-b753-1ccf1d17dd90","Type":"ContainerStarted","Data":"38209be6ec88eacfc2b4045880cf486f6a8310483aa10646ff8ea88b0b133c3e"}
Nov 02 15:27:43.150920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:43.150888    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:27:43.151364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:43.151032    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:27:43.470663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:43.470626    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" event={"ID":"1a0cde38-b591-4c62-b753-1ccf1d17dd90","Type":"ContainerStarted","Data":"97bc08dbefaf99a3cbc56ad997cfc763f2478a3e3de66be4fd9093241db5f988"}
Nov 02 15:27:43.470882 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:43.470731    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:43.590862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:43.590802    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" podStartSLOduration=2.654404453 podStartE2EDuration="3.590782496s" totalImagesPullingTime="936.378043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:43.58777638 +0000 UTC m=+1870.025678512" watchObservedRunningTime="2026-11-02 15:27:43.590782496 +0000 UTC m=+1870.028684675"
Nov 02 15:27:44.669351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.669310    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:27:44.675678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.675657    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:44.711748 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.711706    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:27:44.814214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.778721    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-468af4-dockercfg-rvgp9\""
Nov 02 15:27:44.814214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.779538    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-468af4\""
Nov 02 15:27:44.964312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.917113    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld4nc\" (UniqueName: \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:44.999711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:44.999671    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.035518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.033504    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:27:45.055930 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.055897    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.057529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.057504    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:27:45.111099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.111062    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.111313 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.111291    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld4nc\" (UniqueName: \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.111782 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.111760    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.175271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.175240    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld4nc\" (UniqueName: \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\") pod \"payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.213030 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.212997    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.213241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.213142    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwng\" (UniqueName: \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.311626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.311527    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:45.315906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.315616    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.315906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.315713    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwng\" (UniqueName: \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.317076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.317038    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.376804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.376768    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwng\" (UniqueName: \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\") pod \"payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") " pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.671547 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.671453    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:45.716951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:45.716888    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:27:45.720050 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:27:45.720023    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4c4fe7_8695_4c60_aac9_b7acd1ef5432.slice/crio-a5c5120984a0d109e1bedca9645949ed6a51e42720a79c0120487c4e5874ac64 WatchSource:0}: Error finding container a5c5120984a0d109e1bedca9645949ed6a51e42720a79c0120487c4e5874ac64: Status 404 returned error can't find the container with id a5c5120984a0d109e1bedca9645949ed6a51e42720a79c0120487c4e5874ac64
Nov 02 15:27:46.070998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.070967    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:27:46.071968 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:27:46.071940    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcc35a0_0e4c_4457_8c2a_aaca2a928cf5.slice/crio-3ddd5445221f8cb8a0b040d7f1bd15a6e9484ffa65970b4cba7ecfb75cbf29a5 WatchSource:0}: Error finding container 3ddd5445221f8cb8a0b040d7f1bd15a6e9484ffa65970b4cba7ecfb75cbf29a5: Status 404 returned error can't find the container with id 3ddd5445221f8cb8a0b040d7f1bd15a6e9484ffa65970b4cba7ecfb75cbf29a5
Nov 02 15:27:46.501453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.501412    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" event={"ID":"af4c4fe7-8695-4c60-aac9-b7acd1ef5432","Type":"ContainerStarted","Data":"4c3ab10674a616e4c6c3f979a6ebb9a25bcf60b67054c9b25f80340bb7d3a1de"}
Nov 02 15:27:46.501670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.501484    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" event={"ID":"af4c4fe7-8695-4c60-aac9-b7acd1ef5432","Type":"ContainerStarted","Data":"a5c5120984a0d109e1bedca9645949ed6a51e42720a79c0120487c4e5874ac64"}
Nov 02 15:27:46.501670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.501519    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:46.503074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.503049    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" event={"ID":"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5","Type":"ContainerStarted","Data":"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"}
Nov 02 15:27:46.503218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.503093    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:46.503218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.503110    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" event={"ID":"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5","Type":"ContainerStarted","Data":"3ddd5445221f8cb8a0b040d7f1bd15a6e9484ffa65970b4cba7ecfb75cbf29a5"}
Nov 02 15:27:46.624231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.623173    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" podStartSLOduration=2.6231542389999998 podStartE2EDuration="2.623154239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:46.620654414 +0000 UTC m=+1873.058556580" watchObservedRunningTime="2026-11-02 15:27:46.623154239 +0000 UTC m=+1873.061056382"
Nov 02 15:27:46.747214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:46.743767    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" podStartSLOduration=2.743753254 podStartE2EDuration="2.743753254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:27:46.741734542 +0000 UTC m=+1873.179636685" watchObservedRunningTime="2026-11-02 15:27:46.743753254 +0000 UTC m=+1873.181655455"
Nov 02 15:27:49.499246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:49.499172    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:27:57.539210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:57.538981    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:27:57.547260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:57.540564    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:27:58.602636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.602601    4470 generic.go:356] "Generic (PLEG): container finished" podID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerID="d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b" exitCode=0
Nov 02 15:27:58.603060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.602672    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerDied","Data":"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"}
Nov 02 15:27:58.603133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.603117    4470 scope.go:122] "RemoveContainer" containerID="d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"
Nov 02 15:27:58.604962 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.604932    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:58.605025 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.605012    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:58.605084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:58.605069    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:27:59.612742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:59.612698    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerStarted","Data":"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"}
Nov 02 15:27:59.644537 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:59.644500    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:27:59.646055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:59.646024    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:27:59.646174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:27:59.646072    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:00.645088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:00.645048    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:28:00.645626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:00.645107    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:01.639417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:01.639378    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:28:01.645261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:01.645233    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:28:01.645610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:01.645288    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:02.646801 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:02.646751    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:28:02.647523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:02.647485    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:03.567745 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:03.567703    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:28:03.644386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:03.644347    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:28:03.644583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:03.644423    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:04.645369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:04.645320    4470 patch_prober.go:29] interesting pod/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused" start-of-body=
Nov 02 15:28:04.645825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:04.645385    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.111:15021/healthz/ready\": dial tcp 10.133.0.111:15021: connect: connection refused"
Nov 02 15:28:05.650049 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:05.650010    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:28:05.651168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:05.651144    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:28:16.963168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:16.963128    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:28:16.969002 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:16.968978    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.068336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.068293    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hdjl\" (UniqueName: \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.068649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.068624    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.068728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.068676    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.068819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.068803    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.087398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.087361    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-d6b9c6-serving-cert\""
Nov 02 15:28:17.099976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.099891    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:28:17.199224 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.175415    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.199540    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.204319    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.204468    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.204499    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hdjl\" (UniqueName: \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:17.179356    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-serving-cert: secret "maas-api-e2e-shared-b-w2-d6b9c6-serving-cert" not found
Nov 02 15:28:17.205256 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:17.204844    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls podName:5b22ca92-967c-4aef-94af-7ce2f7fd017d nodeName:}" failed. No retries permitted until 2026-11-02 15:28:17.704818843 +0000 UTC m=+1904.142720975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls") pod "maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" (UID: "5b22ca92-967c-4aef-94af-7ce2f7fd017d") : secret "maas-api-e2e-shared-b-w2-d6b9c6-serving-cert" not found
Nov 02 15:28:17.210798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.210748    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.295916 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.281635    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hdjl\" (UniqueName: \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.729683 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.729639    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.732320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.732291    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:17.909932 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:17.909896    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:18.322917 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:18.322888    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:28:18.324550 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:28:18.324525    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b22ca92_967c_4aef_94af_7ce2f7fd017d.slice/crio-797102c40c0c687264be1d74674bcbc97d152995ac4fa40b910d8c44b4291862 WatchSource:0}: Error finding container 797102c40c0c687264be1d74674bcbc97d152995ac4fa40b910d8c44b4291862: Status 404 returned error can't find the container with id 797102c40c0c687264be1d74674bcbc97d152995ac4fa40b910d8c44b4291862
Nov 02 15:28:18.326781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:18.326762    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:28:18.809561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:18.809522    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" event={"ID":"5b22ca92-967c-4aef-94af-7ce2f7fd017d","Type":"ContainerStarted","Data":"797102c40c0c687264be1d74674bcbc97d152995ac4fa40b910d8c44b4291862"}
Nov 02 15:28:19.821546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:19.821504    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" event={"ID":"5b22ca92-967c-4aef-94af-7ce2f7fd017d","Type":"ContainerStarted","Data":"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"}
Nov 02 15:28:19.822180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:19.822151    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:19.941723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:19.941664    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" podStartSLOduration=2.9750947119999998 podStartE2EDuration="3.941647042s" totalImagesPullingTime="966.55233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:28:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:28:19.939003506 +0000 UTC m=+1906.376905646" watchObservedRunningTime="2026-11-02 15:28:19.941647042 +0000 UTC m=+1906.379549183"
Nov 02 15:28:22.022298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.022261    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:28:22.025631 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.025608    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.077879 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.077848    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-885c69-serving-cert\""
Nov 02 15:28:22.138468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.138428    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:28:22.142627 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.142575    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.142948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.142919    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.143094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.143075    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkzf2\" (UniqueName: \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.143363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.143340    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.245394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.245308    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.245394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.245347    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkzf2\" (UniqueName: \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.245606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.245459    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.245606 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:22.245471    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-serving-cert: secret "maas-api-e2e-shared-b-w5-885c69-serving-cert" not found
Nov 02 15:28:22.245606 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:22.245546    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls podName:970598d6-a53d-4128-9043-8af35fead769 nodeName:}" failed. No retries permitted until 2026-11-02 15:28:22.745526035 +0000 UTC m=+1909.183428168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls") pod "maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" (UID: "970598d6-a53d-4128-9043-8af35fead769") : secret "maas-api-e2e-shared-b-w5-885c69-serving-cert" not found
Nov 02 15:28:22.245768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.245720    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.246544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.246515    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.247991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.247971    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.336709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.336477    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkzf2\" (UniqueName: \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.771440 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.771399    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.773922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.773901    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:22.847891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.847854    4470 generic.go:356] "Generic (PLEG): container finished" podID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerID="3143030bf4f3ac03d02943ea1dc7fde09ca407b8193d61a3011fe9e3e7ac5612" exitCode=0
Nov 02 15:28:22.848070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.847899    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerDied","Data":"3143030bf4f3ac03d02943ea1dc7fde09ca407b8193d61a3011fe9e3e7ac5612"}
Nov 02 15:28:22.848419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.848400    4470 scope.go:122] "RemoveContainer" containerID="3143030bf4f3ac03d02943ea1dc7fde09ca407b8193d61a3011fe9e3e7ac5612"
Nov 02 15:28:22.850393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.850353    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:22.850497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.850435    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:22.850497 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.850483    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:22.939531 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:22.939505    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:23.368986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:23.368954    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:28:23.371074 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:28:23.371046    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970598d6_a53d_4128_9043_8af35fead769.slice/crio-1f7a3adf0fc6d0172c4118dc530e1967236fedcbe1ca30ace583bd880c8f9dec WatchSource:0}: Error finding container 1f7a3adf0fc6d0172c4118dc530e1967236fedcbe1ca30ace583bd880c8f9dec: Status 404 returned error can't find the container with id 1f7a3adf0fc6d0172c4118dc530e1967236fedcbe1ca30ace583bd880c8f9dec
Nov 02 15:28:23.856518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:23.856478    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" event={"ID":"970598d6-a53d-4128-9043-8af35fead769","Type":"ContainerStarted","Data":"1f7a3adf0fc6d0172c4118dc530e1967236fedcbe1ca30ace583bd880c8f9dec"}
Nov 02 15:28:23.858584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:23.858555    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerStarted","Data":"9c73dc6bda20f23c0a668b3d6b9d757f3f44fa63ea0aa89b9e60a1083b962bf4"}
Nov 02 15:28:24.013891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.013856    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:28:24.015061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.015030    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:24.015214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.015093    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:24.868434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.868387    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" event={"ID":"970598d6-a53d-4128-9043-8af35fead769","Type":"ContainerStarted","Data":"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"}
Nov 02 15:28:24.868983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.868469    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:24.997101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:24.997016    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" podStartSLOduration=3.110573534 podStartE2EDuration="3.99699368s" totalImagesPullingTime="886.420146ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:28:24.99656056 +0000 UTC m=+1911.434462725" watchObservedRunningTime="2026-11-02 15:28:24.99699368 +0000 UTC m=+1911.434895822"
Nov 02 15:28:25.013940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:25.013899    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:25.014107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:25.013954    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:25.021409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:25.021313    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:28:25.836542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:25.836513    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:28:26.013417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:26.013378    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:26.013957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:26.013440    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:27.013830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:27.013795    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:27.014394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:27.013851    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:28.013569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:28.013530    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:28.013771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:28.013593    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:29.013767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:29.013722    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused" start-of-body=
Nov 02 15:28:29.014179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:29.013799    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.116:15021/healthz/ready\": dial tcp 10.133.0.116:15021: connect: connection refused"
Nov 02 15:28:30.019011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:30.018978    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:28:30.020138 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:30.020114    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:28:30.891752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:30.891721    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:28:33.566163 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:33.566124    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:28:47.962420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:47.962383    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:28:48.085566 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:48.085533    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:28:51.110259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.110220    4470 generic.go:356] "Generic (PLEG): container finished" podID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerID="3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550" exitCode=0
Nov 02 15:28:51.110844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.110291    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerDied","Data":"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"}
Nov 02 15:28:51.110933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.110914    4470 scope.go:122] "RemoveContainer" containerID="3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"
Nov 02 15:28:51.112844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.112815    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:51.112924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.112891    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:51.112961 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:51.112922    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:28:52.119276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:52.119237    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerStarted","Data":"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"}
Nov 02 15:28:52.146364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:52.146327    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:28:52.146584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:52.146558    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:52.146691 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:52.146632    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:52.963530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:52.963478    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" podUID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" containerName="payload-processing" containerID="cri-o://6a623d5fd941caa9526acfcd5b6cb8a048ddb8465fd375b75be49a63630154f4" gracePeriod=25
Nov 02 15:28:53.091530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.091283    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" podUID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" containerName="payload-pre-processing" containerID="cri-o://2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94" gracePeriod=25
Nov 02 15:28:53.128855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.128821    4470 generic.go:356] "Generic (PLEG): container finished" podID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" containerID="6a623d5fd941caa9526acfcd5b6cb8a048ddb8465fd375b75be49a63630154f4" exitCode=0
Nov 02 15:28:53.129272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.128895    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" event={"ID":"d18ef7b1-7eb8-4bec-844e-f05421f90c90","Type":"ContainerDied","Data":"6a623d5fd941caa9526acfcd5b6cb8a048ddb8465fd375b75be49a63630154f4"}
Nov 02 15:28:53.146658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.146615    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:53.146855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.146689    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:53.327135 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.327109    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:28:53.330707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.330635    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\" (UniqueName: \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\") pod \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") "
Nov 02 15:28:53.331265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.330894    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\") pod \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\" (UID: \"d18ef7b1-7eb8-4bec-844e-f05421f90c90\") "
Nov 02 15:28:53.331265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.331231    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume" pod "d18ef7b1-7eb8-4bec-844e-f05421f90c90" (UID: "d18ef7b1-7eb8-4bec-844e-f05421f90c90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:28:53.331705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.331685    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d18ef7b1-7eb8-4bec-844e-f05421f90c90-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:28:53.340237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.333718    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6" pod "d18ef7b1-7eb8-4bec-844e-f05421f90c90" (UID: "d18ef7b1-7eb8-4bec-844e-f05421f90c90"). InnerVolumeSpecName "kube-api-access-j26d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:28:53.431840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.431810    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:28:53.432746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.432722    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j26d6\" (UniqueName: \"kubernetes.io/projected/d18ef7b1-7eb8-4bec-844e-f05421f90c90-kube-api-access-j26d6\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:28:53.534131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.534034    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\" (UniqueName: \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\") pod \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") "
Nov 02 15:28:53.534330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.534268    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\") pod \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\" (UID: \"9b518da0-159f-4c5d-a4f1-98f2f3db0900\") "
Nov 02 15:28:53.534642 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.534619    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume" pod "9b518da0-159f-4c5d-a4f1-98f2f3db0900" (UID: "9b518da0-159f-4c5d-a4f1-98f2f3db0900"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:28:53.534888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.534869    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b518da0-159f-4c5d-a4f1-98f2f3db0900-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:28:53.536149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.536127    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j" pod "9b518da0-159f-4c5d-a4f1-98f2f3db0900" (UID: "9b518da0-159f-4c5d-a4f1-98f2f3db0900"). InnerVolumeSpecName "kube-api-access-jxg8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:28:53.636405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:53.636360    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxg8j\" (UniqueName: \"kubernetes.io/projected/9b518da0-159f-4c5d-a4f1-98f2f3db0900-kube-api-access-jxg8j\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:28:54.139654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.139619    4470 generic.go:356] "Generic (PLEG): container finished" podID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" containerID="2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94" exitCode=0
Nov 02 15:28:54.140114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.139669    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"
Nov 02 15:28:54.140114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.139700    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" event={"ID":"9b518da0-159f-4c5d-a4f1-98f2f3db0900","Type":"ContainerDied","Data":"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"}
Nov 02 15:28:54.140114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.139759    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg" event={"ID":"9b518da0-159f-4c5d-a4f1-98f2f3db0900","Type":"ContainerDied","Data":"43290182393ec2683734e2351df175ae5ecb0e074133146234d75051a5d9d0a8"}
Nov 02 15:28:54.140114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.139785    4470 scope.go:122] "RemoveContainer" containerID="2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"
Nov 02 15:28:54.141734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.141713    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"
Nov 02 15:28:54.141734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.141715    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9" event={"ID":"d18ef7b1-7eb8-4bec-844e-f05421f90c90","Type":"ContainerDied","Data":"0f0b38a367ad1ba5e60bf6086a4eeae29c7c5d83d466ebf6a594aeaa551f2bc2"}
Nov 02 15:28:54.145913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.145890    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:54.146049 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.145944    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:54.152717 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.152692    4470 scope.go:122] "RemoveContainer" containerID="2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"
Nov 02 15:28:54.153065 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:28:54.153042    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94\": container with ID starting with 2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94 not found: ID does not exist" containerID="2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"
Nov 02 15:28:54.153183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.153080    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94"} err="failed to get container status \"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94\": rpc error: code = NotFound desc = could not find container \"2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94\": container with ID starting with 2c01f4e6a93514b768f495323f7074bdb321749783f7c6aafcf558aadfcacb94 not found: ID does not exist"
Nov 02 15:28:54.153183 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.153109    4470 scope.go:122] "RemoveContainer" containerID="6a623d5fd941caa9526acfcd5b6cb8a048ddb8465fd375b75be49a63630154f4"
Nov 02 15:28:54.309378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.309278    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:28:54.313469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.313446    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-eafc0d-5588fc499c-hsfm9"]
Nov 02 15:28:54.424349 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.424311    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:28:54.428145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:54.428118    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-eafc0d-75789b677f-q7qrg"]
Nov 02 15:28:55.146271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:55.146241    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:55.146742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:55.146292    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:55.723470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:55.723432    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" path="/var/lib/kubelet/pods/9b518da0-159f-4c5d-a4f1-98f2f3db0900/volumes"
Nov 02 15:28:55.724048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:55.724032    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" path="/var/lib/kubelet/pods/d18ef7b1-7eb8-4bec-844e-f05421f90c90/volumes"
Nov 02 15:28:56.145969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:56.145882    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:56.145969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:56.145939    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:57.145964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:57.145926    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Nov 02 15:28:57.146403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:57.145984    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Nov 02 15:28:58.151470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:58.151439    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:28:58.151963 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:58.151943    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:28:58.152733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:28:58.152714    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:29:03.567693 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:03.567652    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:29:06.464458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.464420    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:29:06.465225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465209    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" containerName="payload-processing"
Nov 02 15:29:06.465225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465226    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" containerName="payload-processing"
Nov 02 15:29:06.465340 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465253    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" containerName="payload-pre-processing"
Nov 02 15:29:06.465340 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465260    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" containerName="payload-pre-processing"
Nov 02 15:29:06.465431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465422    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d18ef7b1-7eb8-4bec-844e-f05421f90c90" containerName="payload-processing"
Nov 02 15:29:06.465466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.465437    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b518da0-159f-4c5d-a4f1-98f2f3db0900" containerName="payload-pre-processing"
Nov 02 15:29:06.468114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.468096    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.567880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.567839    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.568072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.567918    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.568072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.567995    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbnsr\" (UniqueName: \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.568286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.568255    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.570223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.570177    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-303ba8-serving-cert\""
Nov 02 15:29:06.576998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.576962    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:29:06.669902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.669869    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.670115 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.669947    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.670115 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.670001    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbnsr\" (UniqueName: \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.670115 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:06.670047    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-serving-cert: secret "maas-api-e2e-api-a-303ba8-serving-cert" not found
Nov 02 15:29:06.670346 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:06.670122    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls podName:2d116e99-553b-4ae3-a03b-d7582826b314 nodeName:}" failed. No retries permitted until 2026-11-02 15:29:07.170096424 +0000 UTC m=+1953.607998560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls") pod "maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" (UID: "2d116e99-553b-4ae3-a03b-d7582826b314") : secret "maas-api-e2e-api-a-303ba8-serving-cert" not found
Nov 02 15:29:06.670346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.670255    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.670832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.670807    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.672845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.672822    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:06.743102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:06.743024    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbnsr\" (UniqueName: \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:07.186841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:07.186803    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:07.189341 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:07.189318    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") pod \"maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:07.382176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:07.382139    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:07.767840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:07.767812    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:29:07.768341 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:29:07.768316    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d116e99_553b_4ae3_a03b_d7582826b314.slice/crio-6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f WatchSource:0}: Error finding container 6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f: Status 404 returned error can't find the container with id 6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f
Nov 02 15:29:08.265513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:08.265474    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" event={"ID":"2d116e99-553b-4ae3-a03b-d7582826b314","Type":"ContainerStarted","Data":"6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f"}
Nov 02 15:29:09.277174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:09.277134    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" event={"ID":"2d116e99-553b-4ae3-a03b-d7582826b314","Type":"ContainerStarted","Data":"0cbf450775e10abf2556058a25ff6a32da119b666d5e324e0aadf9752ccd3393"}
Nov 02 15:29:09.277652 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:09.277252    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:09.396764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:09.396707    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" podStartSLOduration=2.62893663 podStartE2EDuration="3.396691889s" totalImagesPullingTime="767.755259ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:29:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:29:09.394661907 +0000 UTC m=+1955.832564039" watchObservedRunningTime="2026-11-02 15:29:09.396691889 +0000 UTC m=+1955.834594030"
Nov 02 15:29:15.290913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:15.290882    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:29:21.727792 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.727750    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:29:21.735833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.735806    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.790729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.790693    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:29:21.826414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.826359    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g6qk\" (UniqueName: \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.826615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.826595    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.826764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.826745    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.826836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.826807    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.834579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.834544    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-468af4-serving-cert\""
Nov 02 15:29:21.928411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.928370    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g6qk\" (UniqueName: \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.928613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.928498    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.928693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.928621    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.928693 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:21.928663    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-serving-cert: secret "maas-api-e2e-inf-a-468af4-serving-cert" not found
Nov 02 15:29:21.928693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.928676    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.928845 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:21.928729    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls podName:ce766b1b-5caa-44b2-8209-55baa7cb41e6 nodeName:}" failed. No retries permitted until 2026-11-02 15:29:22.428706506 +0000 UTC m=+1968.866608641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls") pod "maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" (UID: "ce766b1b-5caa-44b2-8209-55baa7cb41e6") : secret "maas-api-e2e-inf-a-468af4-serving-cert" not found
Nov 02 15:29:21.929426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.929402    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:21.931439 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:21.931416    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:22.011346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:22.011226    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g6qk\" (UniqueName: \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:22.451033 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:22.450989    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:22.453641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:22.453616    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") pod \"maas-api-e2e-inf-a-468af4-687d6cd55-26v5x\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:22.654765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:22.654728    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:23.042001 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:23.041967    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:29:23.043164 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:29:23.043130    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce766b1b_5caa_44b2_8209_55baa7cb41e6.slice/crio-0b4d1dd1332bc61d447d08bb67180c2fe53aa18c035eb4935b8e931d53cf6d22 WatchSource:0}: Error finding container 0b4d1dd1332bc61d447d08bb67180c2fe53aa18c035eb4935b8e931d53cf6d22: Status 404 returned error can't find the container with id 0b4d1dd1332bc61d447d08bb67180c2fe53aa18c035eb4935b8e931d53cf6d22
Nov 02 15:29:23.392757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:23.392669    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" event={"ID":"ce766b1b-5caa-44b2-8209-55baa7cb41e6","Type":"ContainerStarted","Data":"0b4d1dd1332bc61d447d08bb67180c2fe53aa18c035eb4935b8e931d53cf6d22"}
Nov 02 15:29:24.401737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:24.401625    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" event={"ID":"ce766b1b-5caa-44b2-8209-55baa7cb41e6","Type":"ContainerStarted","Data":"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"}
Nov 02 15:29:24.401737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:24.401698    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:24.524218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:24.524148    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" podStartSLOduration=2.610037842 podStartE2EDuration="3.524130493s" totalImagesPullingTime="914.092651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:29:24.519453696 +0000 UTC m=+1970.957355837" watchObservedRunningTime="2026-11-02 15:29:24.524130493 +0000 UTC m=+1970.962032634"
Nov 02 15:29:26.267251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.267214    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:29:26.267733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.267476    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" podUID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" containerName="maas-api" containerID="cri-o://97bc08dbefaf99a3cbc56ad997cfc763f2478a3e3de66be4fd9093241db5f988" gracePeriod=30
Nov 02 15:29:26.422264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.422222    4470 generic.go:356] "Generic (PLEG): container finished" podID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" containerID="97bc08dbefaf99a3cbc56ad997cfc763f2478a3e3de66be4fd9093241db5f988" exitCode=0
Nov 02 15:29:26.422264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.422243    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" event={"ID":"1a0cde38-b591-4c62-b753-1ccf1d17dd90","Type":"ContainerDied","Data":"97bc08dbefaf99a3cbc56ad997cfc763f2478a3e3de66be4fd9093241db5f988"}
Nov 02 15:29:26.771527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.771446    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:29:26.784261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.783640    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:29:26.801479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.801437    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:29:26.915118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.915030    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\") pod \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") "
Nov 02 15:29:26.915118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.915081    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") pod \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") "
Nov 02 15:29:26.915389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.915124    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\" (UniqueName: \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\") pod \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") "
Nov 02 15:29:26.915449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.915394    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\") pod \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\" (UID: \"1a0cde38-b591-4c62-b753-1ccf1d17dd90\") "
Nov 02 15:29:26.915856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.915506    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca" pod "1a0cde38-b591-4c62-b753-1ccf1d17dd90" (UID: "1a0cde38-b591-4c62-b753-1ccf1d17dd90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:29:26.916418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.916128    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a0cde38-b591-4c62-b753-1ccf1d17dd90-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:26.920638 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.920603    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw" pod "1a0cde38-b591-4c62-b753-1ccf1d17dd90" (UID: "1a0cde38-b591-4c62-b753-1ccf1d17dd90"). InnerVolumeSpecName "kube-api-access-f5xxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:29:26.922418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.922377    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls" pod "1a0cde38-b591-4c62-b753-1ccf1d17dd90" (UID: "1a0cde38-b591-4c62-b753-1ccf1d17dd90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:29:26.924415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:26.924380    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs" pod "1a0cde38-b591-4c62-b753-1ccf1d17dd90" (UID: "1a0cde38-b591-4c62-b753-1ccf1d17dd90"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:29:27.017966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.017923    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:27.017966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.017963    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5xxw\" (UniqueName: \"kubernetes.io/projected/1a0cde38-b591-4c62-b753-1ccf1d17dd90-kube-api-access-f5xxw\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:27.018230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.017978    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a0cde38-b591-4c62-b753-1ccf1d17dd90-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:27.534608 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.534577    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"
Nov 02 15:29:27.534608 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.534592    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc" event={"ID":"1a0cde38-b591-4c62-b753-1ccf1d17dd90","Type":"ContainerDied","Data":"38209be6ec88eacfc2b4045880cf486f6a8310483aa10646ff8ea88b0b133c3e"}
Nov 02 15:29:27.535156 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.534656    4470 scope.go:122] "RemoveContainer" containerID="97bc08dbefaf99a3cbc56ad997cfc763f2478a3e3de66be4fd9093241db5f988"
Nov 02 15:29:27.708761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.708725    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:29:27.750481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:27.750443    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-53fd96-648dd9cfbd-wcvmc"]
Nov 02 15:29:29.765824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:29.765782    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" path="/var/lib/kubelet/pods/1a0cde38-b591-4c62-b753-1ccf1d17dd90/volumes"
Nov 02 15:29:30.460418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:30.460385    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:29:31.789316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:31.789268    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" podUID="d78b2172-9204-4c57-89e4-b390cd1a14ef" containerName="payload-processing" containerID="cri-o://288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584" gracePeriod=25
Nov 02 15:29:31.809929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:31.805280    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" podUID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" containerName="payload-pre-processing" containerID="cri-o://a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9" gracePeriod=25
Nov 02 15:29:32.346210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.346115    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:29:32.354735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.354678    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\" (UniqueName: \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\") pod \"d78b2172-9204-4c57-89e4-b390cd1a14ef\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") "
Nov 02 15:29:32.359182 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.359044    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk" pod "d78b2172-9204-4c57-89e4-b390cd1a14ef" (UID: "d78b2172-9204-4c57-89e4-b390cd1a14ef"). InnerVolumeSpecName "kube-api-access-wxtgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:29:32.446307 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.446275    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:29:32.456011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.455979    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\") pod \"d78b2172-9204-4c57-89e4-b390cd1a14ef\" (UID: \"d78b2172-9204-4c57-89e4-b390cd1a14ef\") "
Nov 02 15:29:32.456163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456034    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\") pod \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") "
Nov 02 15:29:32.456163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456141    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\" (UniqueName: \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\") pod \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\" (UID: \"d4b4f1f0-baa2-4f05-84af-f9ec411464a2\") "
Nov 02 15:29:32.456526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456495    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume" pod "d78b2172-9204-4c57-89e4-b390cd1a14ef" (UID: "d78b2172-9204-4c57-89e4-b390cd1a14ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:29:32.456662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456524    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume" pod "d4b4f1f0-baa2-4f05-84af-f9ec411464a2" (UID: "d4b4f1f0-baa2-4f05-84af-f9ec411464a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:29:32.456710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456664    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:32.456710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456677    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxtgk\" (UniqueName: \"kubernetes.io/projected/d78b2172-9204-4c57-89e4-b390cd1a14ef-kube-api-access-wxtgk\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:32.456710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.456689    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d78b2172-9204-4c57-89e4-b390cd1a14ef-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:32.458357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.458336    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt" pod "d4b4f1f0-baa2-4f05-84af-f9ec411464a2" (UID: "d4b4f1f0-baa2-4f05-84af-f9ec411464a2"). InnerVolumeSpecName "kube-api-access-txbbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:29:32.558296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.558252    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txbbt\" (UniqueName: \"kubernetes.io/projected/d4b4f1f0-baa2-4f05-84af-f9ec411464a2-kube-api-access-txbbt\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:32.732308 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.732268    4470 generic.go:356] "Generic (PLEG): container finished" podID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" containerID="a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9" exitCode=0
Nov 02 15:29:32.732493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.732378    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" event={"ID":"d4b4f1f0-baa2-4f05-84af-f9ec411464a2","Type":"ContainerDied","Data":"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"}
Nov 02 15:29:32.732493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.732456    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284" event={"ID":"d4b4f1f0-baa2-4f05-84af-f9ec411464a2","Type":"ContainerDied","Data":"19321231173f3b114ff52b678e1c6e2fec21a32ab57da604bc85a57a2a5428a8"}
Nov 02 15:29:32.732624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.732495    4470 scope.go:122] "RemoveContainer" containerID="a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"
Nov 02 15:29:32.732893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.732669    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"
Nov 02 15:29:32.737415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.737158    4470 generic.go:356] "Generic (PLEG): container finished" podID="d78b2172-9204-4c57-89e4-b390cd1a14ef" containerID="288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584" exitCode=0
Nov 02 15:29:32.737415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.737323    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" event={"ID":"d78b2172-9204-4c57-89e4-b390cd1a14ef","Type":"ContainerDied","Data":"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"}
Nov 02 15:29:32.737415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.737370    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m" event={"ID":"d78b2172-9204-4c57-89e4-b390cd1a14ef","Type":"ContainerDied","Data":"f3e0cbb968777ca3f03bc904c475c13c518a33fe4abd02c3244fc0ab55aab845"}
Nov 02 15:29:32.737750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.737419    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"
Nov 02 15:29:32.748111 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.748090    4470 scope.go:122] "RemoveContainer" containerID="a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"
Nov 02 15:29:32.748483 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:32.748461    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9\": container with ID starting with a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9 not found: ID does not exist" containerID="a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"
Nov 02 15:29:32.748584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.748489    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9"} err="failed to get container status \"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9\": rpc error: code = NotFound desc = could not find container \"a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9\": container with ID starting with a2f304e1f8efe6d2b4dcc0c0ba39f7797009947450c70d33a05461c29fceafe9 not found: ID does not exist"
Nov 02 15:29:32.748584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.748509    4470 scope.go:122] "RemoveContainer" containerID="288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"
Nov 02 15:29:32.759948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.759929    4470 scope.go:122] "RemoveContainer" containerID="288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"
Nov 02 15:29:32.760225 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:32.760183    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584\": container with ID starting with 288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584 not found: ID does not exist" containerID="288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"
Nov 02 15:29:32.760295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.760234    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584"} err="failed to get container status \"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584\": rpc error: code = NotFound desc = could not find container \"288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584\": container with ID starting with 288e37a2d8e8dd321aec4aa04333810f2f02b5d0ffbcb323bb98fde30c586584 not found: ID does not exist"
Nov 02 15:29:32.939236 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.938345    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:29:32.943211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:32.943070    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-53fd96-c945bcf7-xd78m"]
Nov 02 15:29:33.045377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:33.045290    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:29:33.050220 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:33.049373    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-53fd96-69848f4575-29284"]
Nov 02 15:29:33.568544 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:29:33.568500    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:29:33.728895 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:33.728854    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" path="/var/lib/kubelet/pods/d4b4f1f0-baa2-4f05-84af-f9ec411464a2/volumes"
Nov 02 15:29:33.729429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:33.729405    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d78b2172-9204-4c57-89e4-b390cd1a14ef" path="/var/lib/kubelet/pods/d78b2172-9204-4c57-89e4-b390cd1a14ef/volumes"
Nov 02 15:29:40.546347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.546312    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": read tcp 10.133.0.2:33720->10.133.0.125:15021: read: connection reset by peer" start-of-body=
Nov 02 15:29:40.546797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.546381    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": read tcp 10.133.0.2:33720->10.133.0.125:15021: read: connection reset by peer"
Nov 02 15:29:40.610748 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.610700    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:29:40.611065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.611034    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" containerID="cri-o://9c73dc6bda20f23c0a668b3d6b9d757f3f44fa63ea0aa89b9e60a1083b962bf4" gracePeriod=30
Nov 02 15:29:40.902796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.902680    4470 generic.go:356] "Generic (PLEG): container finished" podID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerID="8c637e38f2ab3d17a6b9b3a3fb8b45674970c5b50aacee87c83731503a812573" exitCode=0
Nov 02 15:29:40.902796 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.902739    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerDied","Data":"8c637e38f2ab3d17a6b9b3a3fb8b45674970c5b50aacee87c83731503a812573"}
Nov 02 15:29:40.903586 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.903561    4470 scope.go:122] "RemoveContainer" containerID="8c637e38f2ab3d17a6b9b3a3fb8b45674970c5b50aacee87c83731503a812573"
Nov 02 15:29:40.906598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.906548    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:40.906705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.906629    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:40.906751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.906670    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:40.907532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.907506    4470 generic.go:356] "Generic (PLEG): container finished" podID="6916f781-7088-43a3-811c-ab761ae73391" containerID="fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142" exitCode=0
Nov 02 15:29:40.907634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.907614    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerDied","Data":"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"}
Nov 02 15:29:40.908168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.908151    4470 scope.go:122] "RemoveContainer" containerID="fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"
Nov 02 15:29:40.911202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.911153    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:40.911306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.911259    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:40.911368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:40.911307    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:29:41.926262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:41.926226    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerStarted","Data":"6dfb19632a434e83340781cb0ab879c69f304877a361c913942e1beae859993f"}
Nov 02 15:29:41.928751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:41.928726    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerStarted","Data":"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"}
Nov 02 15:29:42.511212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.510710    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:29:42.511212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.510990    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Nov 02 15:29:42.511212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.511061    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Nov 02 15:29:42.520528 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.516479    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:29:42.522007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.521922    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Nov 02 15:29:42.522169 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:42.522002    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Nov 02 15:29:43.510213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:43.509613    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Nov 02 15:29:43.510213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:43.509790    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Nov 02 15:29:43.518212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:43.515351    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Nov 02 15:29:43.518212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:43.515408    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Nov 02 15:29:44.508827 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:44.508791    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Nov 02 15:29:44.509086 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:44.509055    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Nov 02 15:29:44.517617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:44.517584    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Nov 02 15:29:44.518237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:44.518170    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Nov 02 15:29:44.534223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:44.534172    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:29:45.023157 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.023110    4470 patch_prober.go:29] interesting pod/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:29:45.023524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.023456    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:29:45.510682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.510638    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Nov 02 15:29:45.511008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.510977    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Nov 02 15:29:45.518664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.518626    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Nov 02 15:29:45.519334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.519302    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Nov 02 15:29:45.968941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.968908    4470 generic.go:356] "Generic (PLEG): container finished" podID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerID="9c73dc6bda20f23c0a668b3d6b9d757f3f44fa63ea0aa89b9e60a1083b962bf4" exitCode=0
Nov 02 15:29:45.969127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.969013    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerDied","Data":"9c73dc6bda20f23c0a668b3d6b9d757f3f44fa63ea0aa89b9e60a1083b962bf4"}
Nov 02 15:29:45.969127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:45.969058    4470 scope.go:122] "RemoveContainer" containerID="3143030bf4f3ac03d02943ea1dc7fde09ca407b8193d61a3011fe9e3e7ac5612"
Nov 02 15:29:46.513274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.512810    4470 patch_prober.go:29] interesting pod/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused" start-of-body=
Nov 02 15:29:46.513274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.512873    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.125:15021/healthz/ready\": dial tcp 10.133.0.125:15021: connect: connection refused"
Nov 02 15:29:46.521847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.521808    4470 patch_prober.go:29] interesting pod/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused" start-of-body=
Nov 02 15:29:46.522546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.521884    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.120:15021/healthz/ready\": dial tcp 10.133.0.120:15021: connect: connection refused"
Nov 02 15:29:46.531643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.531620    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:29:46.560858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.560828    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.560920    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.560981    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561077 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561002    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561087    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561126    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561136    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:29:46.561180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561168    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561173    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:29:46.561551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561510    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:29:46.561551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561536    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.561716 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561570    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\") pod \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\" (UID: \"12f8cb24-efd6-4fbc-b353-86bb64d2c71a\") "
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.561976    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.562274    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.562385    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.562407    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.562421    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.562435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.562436    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.563430 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.563407    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:29:46.563513 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.563429    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "kube-api-access-nfm9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:29:46.563874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.563851    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:29:46.563972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.563906    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy" pod "12f8cb24-efd6-4fbc-b353-86bb64d2c71a" (UID: "12f8cb24-efd6-4fbc-b353-86bb64d2c71a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:29:46.664323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.664288    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.664323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.664321    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.664541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.664338    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfm9w\" (UniqueName: \"kubernetes.io/projected/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-kube-api-access-nfm9w\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.664541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.664353    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:46.664541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:46.664366    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12f8cb24-efd6-4fbc-b353-86bb64d2c71a-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:29:47.023221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.022945    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk" event={"ID":"12f8cb24-efd6-4fbc-b353-86bb64d2c71a","Type":"ContainerDied","Data":"68815de23e37112d84dc8abca7173000ed6f4ef49cec78dea9ccc46a24f21372"}
Nov 02 15:29:47.023221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.023010    4470 scope.go:122] "RemoveContainer" containerID="9c73dc6bda20f23c0a668b3d6b9d757f3f44fa63ea0aa89b9e60a1083b962bf4"
Nov 02 15:29:47.023221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.023111    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"
Nov 02 15:29:47.230968 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.230645    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:29:47.255115 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.255084    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-53fd96-openshift-default-677ff6848c-q4fjk"]
Nov 02 15:29:47.523345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.523313    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:29:47.523941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.523875    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:29:47.524658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.524640    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:29:47.539308 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.539285    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:29:47.540531 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.540511    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:29:47.717386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:47.717351    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" path="/var/lib/kubelet/pods/12f8cb24-efd6-4fbc-b353-86bb64d2c71a/volumes"
Nov 02 15:29:55.617417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:55.617377    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:29:55.617959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:55.617592    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" containerID="cri-o://9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37" gracePeriod=30
Nov 02 15:29:56.609055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:56.609010    4470 patch_prober.go:29] interesting pod/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:29:56.609315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:29:56.609088    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:30:00.895850 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:00.895779    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:30:01.009449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009409    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009499    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009578    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009638    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009715    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009748    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009798    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.009836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009806    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:01.009836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009829    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.010093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009867    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:01.010093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.009990    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:01.010093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.010021    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:01.010093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.010030    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\") pod \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\" (UID: \"8ddc1efd-e33d-44cc-862e-b2b8abee1c20\") "
Nov 02 15:30:01.010418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.010394    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:01.011003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.010974    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.011120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.011006    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.011120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.011024    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.011120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.011038    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.011120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.011059    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.012056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.012027    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:01.012158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.012084    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:30:01.012322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.012303    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:01.012396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.012346    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg" pod "8ddc1efd-e33d-44cc-862e-b2b8abee1c20" (UID: "8ddc1efd-e33d-44cc-862e-b2b8abee1c20"). InnerVolumeSpecName "kube-api-access-hz5kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:01.112893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.112857    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.112893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.112892    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.113114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.112905    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz5kg\" (UniqueName: \"kubernetes.io/projected/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-kube-api-access-hz5kg\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.113114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.112920    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ddc1efd-e33d-44cc-862e-b2b8abee1c20-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:01.268628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.268589    4470 generic.go:356] "Generic (PLEG): container finished" podID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerID="9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37" exitCode=0
Nov 02 15:30:01.269011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.268989    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" event={"ID":"8ddc1efd-e33d-44cc-862e-b2b8abee1c20","Type":"ContainerDied","Data":"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"}
Nov 02 15:30:01.269180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.269164    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq" event={"ID":"8ddc1efd-e33d-44cc-862e-b2b8abee1c20","Type":"ContainerDied","Data":"93a1b2d2223c32aca70b60602a614894a3b67db44203ea6592490e876e9f15a9"}
Nov 02 15:30:01.269385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.269369    4470 scope.go:122] "RemoveContainer" containerID="9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"
Nov 02 15:30:01.269628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.269605    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"
Nov 02 15:30:01.353059 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.353029    4470 scope.go:122] "RemoveContainer" containerID="9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"
Nov 02 15:30:01.371210 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:01.359628    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37\": container with ID starting with 9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37 not found: ID does not exist" containerID="9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"
Nov 02 15:30:01.371210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.359691    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37"} err="failed to get container status \"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37\": rpc error: code = NotFound desc = could not find container \"9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37\": container with ID starting with 9ce8948da6a55d1df04a745714d91afd2317e6d4d3204c17327132640f907d37 not found: ID does not exist"
Nov 02 15:30:01.511726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.506661    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:30:01.520081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.519755    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-lbhwq"]
Nov 02 15:30:01.745212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:01.742972    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" path="/var/lib/kubelet/pods/8ddc1efd-e33d-44cc-862e-b2b8abee1c20/volumes"
Nov 02 15:30:03.566633 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:03.566590    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:30:10.797477 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:10.797444    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:30:10.797870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:10.797644    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" containerID="cri-o://c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5" gracePeriod=30
Nov 02 15:30:10.847741 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:10.847705    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:30:10.847977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:10.847951    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy" containerID="cri-o://cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604" gracePeriod=30
Nov 02 15:30:14.692072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:14.692029    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 15:30:14.692511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:14.692110    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 15:30:15.122795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.122705    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:30:15.123038 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.123004    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" podUID="79e443ad-b25f-4f75-9d0e-0ac310e712da" containerName="maas-api" containerID="cri-o://2f39e6b0abeb345be623900418ee009a415172b0ba35c5ec73075643bbe9ae0f" gracePeriod=30
Nov 02 15:30:15.439493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.439075    4470 generic.go:356] "Generic (PLEG): container finished" podID="79e443ad-b25f-4f75-9d0e-0ac310e712da" containerID="2f39e6b0abeb345be623900418ee009a415172b0ba35c5ec73075643bbe9ae0f" exitCode=0
Nov 02 15:30:15.439493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.439271    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" event={"ID":"79e443ad-b25f-4f75-9d0e-0ac310e712da","Type":"ContainerDied","Data":"2f39e6b0abeb345be623900418ee009a415172b0ba35c5ec73075643bbe9ae0f"}
Nov 02 15:30:15.500040 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.500012    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:30:15.649847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.649807    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") pod \"79e443ad-b25f-4f75-9d0e-0ac310e712da\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") "
Nov 02 15:30:15.650391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.650358    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\") pod \"79e443ad-b25f-4f75-9d0e-0ac310e712da\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") "
Nov 02 15:30:15.650697 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.650578    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\") pod \"79e443ad-b25f-4f75-9d0e-0ac310e712da\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") "
Nov 02 15:30:15.650988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.650967    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\" (UniqueName: \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\") pod \"79e443ad-b25f-4f75-9d0e-0ac310e712da\" (UID: \"79e443ad-b25f-4f75-9d0e-0ac310e712da\") "
Nov 02 15:30:15.655056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.654961    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca" pod "79e443ad-b25f-4f75-9d0e-0ac310e712da" (UID: "79e443ad-b25f-4f75-9d0e-0ac310e712da"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:15.668127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.667381    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls" pod "79e443ad-b25f-4f75-9d0e-0ac310e712da" (UID: "79e443ad-b25f-4f75-9d0e-0ac310e712da"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:30:15.668127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.668085    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs" pod "79e443ad-b25f-4f75-9d0e-0ac310e712da" (UID: "79e443ad-b25f-4f75-9d0e-0ac310e712da"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:30:15.673551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.673486    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw" pod "79e443ad-b25f-4f75-9d0e-0ac310e712da" (UID: "79e443ad-b25f-4f75-9d0e-0ac310e712da"). InnerVolumeSpecName "kube-api-access-vkhrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:15.754032 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.753995    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79e443ad-b25f-4f75-9d0e-0ac310e712da-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:15.754725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.754698    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:15.754851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.754838    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkhrw\" (UniqueName: \"kubernetes.io/projected/79e443ad-b25f-4f75-9d0e-0ac310e712da-kube-api-access-vkhrw\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:15.754936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:15.754923    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79e443ad-b25f-4f75-9d0e-0ac310e712da-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.144212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.100548    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183366    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183420    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183480    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183580    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183620    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183721    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183768    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.183825    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.188783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.184014    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\") pod \"bca3b944-22d2-4b34-921f-eccfb3b1090c\" (UID: \"bca3b944-22d2-4b34-921f-eccfb3b1090c\") "
Nov 02 15:30:16.215239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.208577    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.215239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.208925    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:16.215239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.213789    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.215239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.214559    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.215239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.214586    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.256434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.250821    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.256434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.251222    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "kube-api-access-rjlwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:16.256434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.251478    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:16.256951 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.256906    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo" pod "bca3b944-22d2-4b34-921f-eccfb3b1090c" (UID: "bca3b944-22d2-4b34-921f-eccfb3b1090c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:30:16.285981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.285945    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjlwp\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-kube-api-access-rjlwp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.285981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.285986    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286003    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286016    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286031    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286044    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bca3b944-22d2-4b34-921f-eccfb3b1090c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286057    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286071    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.286173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.286084    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bca3b944-22d2-4b34-921f-eccfb3b1090c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.291069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.291038    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:30:16.387031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.386989    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387082    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387224    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387262    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387323    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387431 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387358    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387483    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387515    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.387590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.387584    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\") pod \"8a378458-a569-41a5-bcc0-98322d53a727\" (UID: \"8a378458-a569-41a5-bcc0-98322d53a727\") "
Nov 02 15:30:16.388249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.388027    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.388380 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.388289    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.398435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.388487    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.398435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.389889    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.398435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.390838    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:16.398435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.397643    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:16.398787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.398760    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:16.399036 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399004    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "kube-api-access-7mpnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399327    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399358    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mpnv\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-kube-api-access-7mpnv\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399374    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399391    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399405    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a378458-a569-41a5-bcc0-98322d53a727-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399418    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a378458-a569-41a5-bcc0-98322d53a727-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399433    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.400598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.399447    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a378458-a569-41a5-bcc0-98322d53a727-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.401402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.401378    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo" pod "8a378458-a569-41a5-bcc0-98322d53a727" (UID: "8a378458-a569-41a5-bcc0-98322d53a727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.450841    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.450865    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb" event={"ID":"79e443ad-b25f-4f75-9d0e-0ac310e712da","Type":"ContainerDied","Data":"78ab1d3561223e3ca9cb6663c3b232220234d4bd028b2f526d37fafbb2398565"}
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.450965    4470 scope.go:122] "RemoveContainer" containerID="2f39e6b0abeb345be623900418ee009a415172b0ba35c5ec73075643bbe9ae0f"
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.457170    4470 generic.go:356] "Generic (PLEG): container finished" podID="8a378458-a569-41a5-bcc0-98322d53a727" containerID="c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5" exitCode=0
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.457269    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.457263    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" event={"ID":"8a378458-a569-41a5-bcc0-98322d53a727","Type":"ContainerDied","Data":"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"}
Nov 02 15:30:16.458521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.457451    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h" event={"ID":"8a378458-a569-41a5-bcc0-98322d53a727","Type":"ContainerDied","Data":"74879a8ca814ce7e67f18f48e8de7b3a0c32d11b71d4910aab1951ff7fcbe843"}
Nov 02 15:30:16.461100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.459405    4470 generic.go:356] "Generic (PLEG): container finished" podID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerID="cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604" exitCode=0
Nov 02 15:30:16.461100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.459457    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" event={"ID":"bca3b944-22d2-4b34-921f-eccfb3b1090c","Type":"ContainerDied","Data":"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"}
Nov 02 15:30:16.461100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.459503    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf" event={"ID":"bca3b944-22d2-4b34-921f-eccfb3b1090c","Type":"ContainerDied","Data":"d4ede90d76e09418587be982306885fbb4f23873d7576e847e0f1197832048dd"}
Nov 02 15:30:16.461100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.459613    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"
Nov 02 15:30:16.468945 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.468765    4470 scope.go:122] "RemoveContainer" containerID="c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"
Nov 02 15:30:16.484087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.484059    4470 scope.go:122] "RemoveContainer" containerID="c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"
Nov 02 15:30:16.484529 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:16.484497    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5\": container with ID starting with c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5 not found: ID does not exist" containerID="c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"
Nov 02 15:30:16.484655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.484540    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5"} err="failed to get container status \"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5\": rpc error: code = NotFound desc = could not find container \"c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5\": container with ID starting with c3da27110b68658384760dab035510e8f14e9e6a3679bbe716b2726aadb34ad5 not found: ID does not exist"
Nov 02 15:30:16.484655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.484567    4470 scope.go:122] "RemoveContainer" containerID="cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"
Nov 02 15:30:16.496629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.496601    4470 scope.go:122] "RemoveContainer" containerID="cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"
Nov 02 15:30:16.496966 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:16.496942    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604\": container with ID starting with cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604 not found: ID does not exist" containerID="cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"
Nov 02 15:30:16.497005 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.496975    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604"} err="failed to get container status \"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604\": rpc error: code = NotFound desc = could not find container \"cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604\": container with ID starting with cb452912dcc925c18770e4a73ca39e00b29adea7b73ba156b653434a132d0604 not found: ID does not exist"
Nov 02 15:30:16.501031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.501009    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a378458-a569-41a5-bcc0-98322d53a727-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:16.698462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.697985    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:30:16.706080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.706030    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eafc0d-75f9b7d6b4-sc6sb"]
Nov 02 15:30:16.775583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.775538    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:30:16.778950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.778899    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-8cg2h"]
Nov 02 15:30:16.893476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.893439    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:30:16.898574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:16.898533    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-pnqhf"]
Nov 02 15:30:17.708511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:17.708480    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79e443ad-b25f-4f75-9d0e-0ac310e712da" path="/var/lib/kubelet/pods/79e443ad-b25f-4f75-9d0e-0ac310e712da/volumes"
Nov 02 15:30:17.709010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:17.708995    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a378458-a569-41a5-bcc0-98322d53a727" path="/var/lib/kubelet/pods/8a378458-a569-41a5-bcc0-98322d53a727/volumes"
Nov 02 15:30:17.709487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:17.709472    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" path="/var/lib/kubelet/pods/bca3b944-22d2-4b34-921f-eccfb3b1090c/volumes"
Nov 02 15:30:25.794797 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:25.794759    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:30:25.795259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:25.795064    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy" containerID="cri-o://49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd" gracePeriod=30
Nov 02 15:30:31.088809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.088776    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:30:31.136311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136279    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136322    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136368    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136426    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136493    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136571    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:31.136630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136588    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:31.136630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136606    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136657    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136826    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:31.136948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136855    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:31.136948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136880    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.136948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.136923    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\") pod \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\" (UID: \"a3ab13b7-1bf8-4a67-86fc-bdb411713174\") "
Nov 02 15:30:31.137656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.137630    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:31.137839 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.137633    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.137986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.137852    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.137986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.137870    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.137986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.137884    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.138975 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.138948    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:30:31.139075 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.138999    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:31.139271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.139247    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "kube-api-access-pvt9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:31.139373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.139361    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy" pod "a3ab13b7-1bf8-4a67-86fc-bdb411713174" (UID: "a3ab13b7-1bf8-4a67-86fc-bdb411713174"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:31.238929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.238895    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.238929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.238925    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvt9p\" (UniqueName: \"kubernetes.io/projected/a3ab13b7-1bf8-4a67-86fc-bdb411713174-kube-api-access-pvt9p\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.238929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.238936    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.239166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.238945    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.239166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.238954    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3ab13b7-1bf8-4a67-86fc-bdb411713174-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:31.591171 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.591135    4470 generic.go:356] "Generic (PLEG): container finished" podID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerID="49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd" exitCode=0
Nov 02 15:30:31.591402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.591223    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"
Nov 02 15:30:31.591402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.591221    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerDied","Data":"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"}
Nov 02 15:30:31.591402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.591362    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l" event={"ID":"a3ab13b7-1bf8-4a67-86fc-bdb411713174","Type":"ContainerDied","Data":"499e16ef9410be49e20793c3ec3bc9b75a18f82f764bd7643afb5af325271640"}
Nov 02 15:30:31.591402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.591384    4470 scope.go:122] "RemoveContainer" containerID="49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"
Nov 02 15:30:31.602944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.602924    4470 scope.go:122] "RemoveContainer" containerID="d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"
Nov 02 15:30:31.612454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.612438    4470 scope.go:122] "RemoveContainer" containerID="49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"
Nov 02 15:30:31.612710 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:31.612694    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd\": container with ID starting with 49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd not found: ID does not exist" containerID="49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"
Nov 02 15:30:31.612792 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.612719    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd"} err="failed to get container status \"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd\": rpc error: code = NotFound desc = could not find container \"49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd\": container with ID starting with 49aa557f983b434e6556b9bf83cb7674c4e8eee4a912aa829e9e1d87a01f67fd not found: ID does not exist"
Nov 02 15:30:31.612792 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.612734    4470 scope.go:122] "RemoveContainer" containerID="d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"
Nov 02 15:30:31.612983 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:31.612964    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b\": container with ID starting with d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b not found: ID does not exist" containerID="d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"
Nov 02 15:30:31.613022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.612989    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b"} err="failed to get container status \"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b\": rpc error: code = NotFound desc = could not find container \"d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b\": container with ID starting with d8d87b6373d3aaa95d3af01a66a9dfb2e90947a501f5e30a9dd9b8a4eb43193b not found: ID does not exist"
Nov 02 15:30:31.757103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.757072    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:30:31.807640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:31.807606    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-eafc0d-openshift-default-78865f7d8c-kxv7l"]
Nov 02 15:30:32.619448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:32.619413    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:30:33.567104 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:33.567068    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:30:33.707588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:33.707553    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" path="/var/lib/kubelet/pods/a3ab13b7-1bf8-4a67-86fc-bdb411713174/volumes"
Nov 02 15:30:37.726558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:37.726524    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:30:37.828851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:37.828672    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:30:42.401321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.401279    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:30:42.401759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.401539    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" podUID="2d116e99-553b-4ae3-a03b-d7582826b314" containerName="maas-api" containerID="cri-o://0cbf450775e10abf2556058a25ff6a32da119b666d5e324e0aadf9752ccd3393" gracePeriod=30
Nov 02 15:30:42.727104 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.727038    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" podUID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" containerName="payload-processing" containerID="cri-o://0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348" gracePeriod=25
Nov 02 15:30:42.842232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.829240    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerName="payload-pre-processing" containerID="cri-o://8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69" gracePeriod=25
Nov 02 15:30:42.880265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.880181    4470 generic.go:356] "Generic (PLEG): container finished" podID="2d116e99-553b-4ae3-a03b-d7582826b314" containerID="0cbf450775e10abf2556058a25ff6a32da119b666d5e324e0aadf9752ccd3393" exitCode=0
Nov 02 15:30:42.880546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.880524    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" event={"ID":"2d116e99-553b-4ae3-a03b-d7582826b314","Type":"ContainerDied","Data":"0cbf450775e10abf2556058a25ff6a32da119b666d5e324e0aadf9752ccd3393"}
Nov 02 15:30:42.880688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.880674    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s" event={"ID":"2d116e99-553b-4ae3-a03b-d7582826b314","Type":"ContainerDied","Data":"6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f"}
Nov 02 15:30:42.880810 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.880794    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e262815f53136c53ab62a0f94557f36b0f6ecc2c4ee2b0ca56f913cc1d1900f"
Nov 02 15:30:42.886302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:42.885586    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:30:43.034243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.031213    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\") pod \"2d116e99-553b-4ae3-a03b-d7582826b314\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") "
Nov 02 15:30:43.034243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.031379    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") pod \"2d116e99-553b-4ae3-a03b-d7582826b314\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") "
Nov 02 15:30:43.034243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.031448    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\") pod \"2d116e99-553b-4ae3-a03b-d7582826b314\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") "
Nov 02 15:30:43.034243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.031508    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\" (UniqueName: \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\") pod \"2d116e99-553b-4ae3-a03b-d7582826b314\" (UID: \"2d116e99-553b-4ae3-a03b-d7582826b314\") "
Nov 02 15:30:43.034243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.034058    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca" pod "2d116e99-553b-4ae3-a03b-d7582826b314" (UID: "2d116e99-553b-4ae3-a03b-d7582826b314"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:43.043939 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.037230    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs" pod "2d116e99-553b-4ae3-a03b-d7582826b314" (UID: "2d116e99-553b-4ae3-a03b-d7582826b314"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:30:43.043939 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.045762    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr" pod "2d116e99-553b-4ae3-a03b-d7582826b314" (UID: "2d116e99-553b-4ae3-a03b-d7582826b314"). InnerVolumeSpecName "kube-api-access-mbnsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:43.059713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.059421    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls" pod "2d116e99-553b-4ae3-a03b-d7582826b314" (UID: "2d116e99-553b-4ae3-a03b-d7582826b314"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:30:43.093912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.093638    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:30:43.133775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.133733    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\") pod \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") "
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134020    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\" (UniqueName: \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\") pod \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\" (UID: \"248ccda4-84a5-4a23-9fb2-2f782e0e34c6\") "
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134354    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume" pod "248ccda4-84a5-4a23-9fb2-2f782e0e34c6" (UID: "248ccda4-84a5-4a23-9fb2-2f782e0e34c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134810    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d116e99-553b-4ae3-a03b-d7582826b314-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134830    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134843    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134851    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d116e99-553b-4ae3-a03b-d7582826b314-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.135372 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.134860    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbnsr\" (UniqueName: \"kubernetes.io/projected/2d116e99-553b-4ae3-a03b-d7582826b314-kube-api-access-mbnsr\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.137247 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.137226    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t" pod "248ccda4-84a5-4a23-9fb2-2f782e0e34c6" (UID: "248ccda4-84a5-4a23-9fb2-2f782e0e34c6"). InnerVolumeSpecName "kube-api-access-d989t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:43.171179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.171152    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:30:43.236028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.235989    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\") pod \"9400a595-d018-4feb-a913-9d2ec496cbf7\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") "
Nov 02 15:30:43.236251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.236094    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\" (UniqueName: \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\") pod \"9400a595-d018-4feb-a913-9d2ec496cbf7\" (UID: \"9400a595-d018-4feb-a913-9d2ec496cbf7\") "
Nov 02 15:30:43.236443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.236416    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume" pod "9400a595-d018-4feb-a913-9d2ec496cbf7" (UID: "9400a595-d018-4feb-a913-9d2ec496cbf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:43.236938 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.236908    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d989t\" (UniqueName: \"kubernetes.io/projected/248ccda4-84a5-4a23-9fb2-2f782e0e34c6-kube-api-access-d989t\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.237080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.236941    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9400a595-d018-4feb-a913-9d2ec496cbf7-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.238408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.238370    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j" pod "9400a595-d018-4feb-a913-9d2ec496cbf7" (UID: "9400a595-d018-4feb-a913-9d2ec496cbf7"). InnerVolumeSpecName "kube-api-access-djm9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:43.338472 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.338395    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djm9j\" (UniqueName: \"kubernetes.io/projected/9400a595-d018-4feb-a913-9d2ec496cbf7-kube-api-access-djm9j\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:43.889063 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.889016    4470 generic.go:356] "Generic (PLEG): container finished" podID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerID="8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69" exitCode=0
Nov 02 15:30:43.889561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.889067    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"
Nov 02 15:30:43.889561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.889092    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" event={"ID":"9400a595-d018-4feb-a913-9d2ec496cbf7","Type":"ContainerDied","Data":"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"}
Nov 02 15:30:43.889561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.889157    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" event={"ID":"9400a595-d018-4feb-a913-9d2ec496cbf7","Type":"ContainerDied","Data":"a645d0552a14f15d15716474686bfe7b4999adc27ab3b295e93e4432671d7222"}
Nov 02 15:30:43.889561 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.889199    4470 scope.go:122] "RemoveContainer" containerID="8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"
Nov 02 15:30:43.890915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.890896    4470 generic.go:356] "Generic (PLEG): container finished" podID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" containerID="0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348" exitCode=0
Nov 02 15:30:43.890999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.890955    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"
Nov 02 15:30:43.890999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.890959    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" event={"ID":"248ccda4-84a5-4a23-9fb2-2f782e0e34c6","Type":"ContainerDied","Data":"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"}
Nov 02 15:30:43.891084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.891034    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2" event={"ID":"248ccda4-84a5-4a23-9fb2-2f782e0e34c6","Type":"ContainerDied","Data":"96ec5e11ce024c8f20f173718c80dd33a903752529f23ca879f939c01ff55357"}
Nov 02 15:30:43.891246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.891222    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"
Nov 02 15:30:43.901490 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.901476    4470 scope.go:122] "RemoveContainer" containerID="8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"
Nov 02 15:30:43.901841 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:43.901816    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69\": container with ID starting with 8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69 not found: ID does not exist" containerID="8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"
Nov 02 15:30:43.901909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.901851    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69"} err="failed to get container status \"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69\": rpc error: code = NotFound desc = could not find container \"8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69\": container with ID starting with 8353620fc6b9337e01df8e5f344b8d7a50347568eac21aa5d1552dd395982f69 not found: ID does not exist"
Nov 02 15:30:43.901909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.901870    4470 scope.go:122] "RemoveContainer" containerID="0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"
Nov 02 15:30:43.911508 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.911480    4470 scope.go:122] "RemoveContainer" containerID="0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"
Nov 02 15:30:43.911783 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:43.911763    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348\": container with ID starting with 0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348 not found: ID does not exist" containerID="0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"
Nov 02 15:30:43.911838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:43.911792    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348"} err="failed to get container status \"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348\": rpc error: code = NotFound desc = could not find container \"0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348\": container with ID starting with 0a097c8497a2d7db1652f0766e612bc3548ca5e022d99152c000485199052348 not found: ID does not exist"
Nov 02 15:30:44.080270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.075732    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:30:44.120705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.120666    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b"]
Nov 02 15:30:44.148446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.148399    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-303ba8-5dc89d9499-zhc4b" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.126:50052\" within 1s: context deadline exceeded"
Nov 02 15:30:44.148772 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:30:44.148494    4470 logging.go:55] [core] [Channel #2249 SubChannel #2250]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.126:50052", ServerName: "10.133.0.126:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.126:50052: operation was canceled"
Nov 02 15:30:44.191119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.191069    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:30:44.195375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.195343    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-303ba8-6d8fbd99d-bdg7s"]
Nov 02 15:30:44.303984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.303931    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:30:44.307806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:44.307777    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-303ba8-7fb5f66b89-trxg2"]
Nov 02 15:30:45.708140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:45.708104    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" path="/var/lib/kubelet/pods/248ccda4-84a5-4a23-9fb2-2f782e0e34c6/volumes"
Nov 02 15:30:45.708686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:45.708668    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d116e99-553b-4ae3-a03b-d7582826b314" path="/var/lib/kubelet/pods/2d116e99-553b-4ae3-a03b-d7582826b314/volumes"
Nov 02 15:30:45.709070 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:45.709057    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" path="/var/lib/kubelet/pods/9400a595-d018-4feb-a913-9d2ec496cbf7/volumes"
Nov 02 15:30:47.642411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.642372    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main" containerID="cri-o://cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db" gracePeriod=45
Nov 02 15:30:47.922618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.922586    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:30:47.923574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.923545    4470 generic.go:356] "Generic (PLEG): container finished" podID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerID="cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db" exitCode=0
Nov 02 15:30:47.923686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.923620    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" event={"ID":"629aba4f-7644-4aec-88d3-f4559b6d3179","Type":"ContainerDied","Data":"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"}
Nov 02 15:30:47.923686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.923675    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626" event={"ID":"629aba4f-7644-4aec-88d3-f4559b6d3179","Type":"ContainerDied","Data":"6b59767bb583842af61ebe136a99dc6982bfc7838e53a67a8da62e3a6b6a45fe"}
Nov 02 15:30:47.923809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.923699    4470 scope.go:122] "RemoveContainer" containerID="cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"
Nov 02 15:30:47.934242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.934222    4470 scope.go:122] "RemoveContainer" containerID="cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"
Nov 02 15:30:47.934542 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:30:47.934520    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db\": container with ID starting with cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db not found: ID does not exist" containerID="cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"
Nov 02 15:30:47.934617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:47.934556    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db"} err="failed to get container status \"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db\": rpc error: code = NotFound desc = could not find container \"cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db\": container with ID starting with cc42730730a831a34337163af7c3f96c2f42ef0ddf406011210c3d2c047868db not found: ID does not exist"
Nov 02 15:30:48.040168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040131    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.040374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040308    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\" (UniqueName: \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.040374 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040354    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.040459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040410    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.040564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040548    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.040647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.040592    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\" (UniqueName: \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\") pod \"629aba4f-7644-4aec-88d3-f4559b6d3179\" (UID: \"629aba4f-7644-4aec-88d3-f4559b6d3179\") "
Nov 02 15:30:48.041575 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.041526    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:48.041776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.041696    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:48.041908 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.041764    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:48.043428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.043396    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "kube-api-access-cwpzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:48.043904 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.043879    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:30:48.044007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.043914    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm" pod "629aba4f-7644-4aec-88d3-f4559b6d3179" (UID: "629aba4f-7644-4aec-88d3-f4559b6d3179"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:48.142725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142684    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.142725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142722    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/629aba4f-7644-4aec-88d3-f4559b6d3179-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.142725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142731    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.143041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142744    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwpzj\" (UniqueName: \"kubernetes.io/projected/629aba4f-7644-4aec-88d3-f4559b6d3179-kube-api-access-cwpzj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.143041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142753    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.143041 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.142763    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/629aba4f-7644-4aec-88d3-f4559b6d3179-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:48.932221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:48.932180    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"
Nov 02 15:30:49.097459 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:49.097421    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:30:49.146806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:49.146767    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-6dc69b/distinct-w4-6dc69b-kserve-6676c8bfdf-hf626"]
Nov 02 15:30:49.707851 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:49.707809    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" path="/var/lib/kubelet/pods/629aba4f-7644-4aec-88d3-f4559b6d3179/volumes"
Nov 02 15:30:51.845570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:51.845533    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:30:51.845975 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:51.845779    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy" containerID="cri-o://6dfb19632a434e83340781cb0ab879c69f304877a361c913942e1beae859993f" gracePeriod=30
Nov 02 15:30:56.654411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:56.654369    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:30:56.765114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:56.765069    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:30:57.019947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.019905    4470 generic.go:356] "Generic (PLEG): container finished" podID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerID="6dfb19632a434e83340781cb0ab879c69f304877a361c913942e1beae859993f" exitCode=0
Nov 02 15:30:57.020161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.019978    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerDied","Data":"6dfb19632a434e83340781cb0ab879c69f304877a361c913942e1beae859993f"}
Nov 02 15:30:57.020161 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.020050    4470 scope.go:122] "RemoveContainer" containerID="8c637e38f2ab3d17a6b9b3a3fb8b45674970c5b50aacee87c83731503a812573"
Nov 02 15:30:57.226806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.226721    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:30:57.314216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.288312    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.314560 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.303548    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:57.314855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.314834    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.315378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.315357    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330320    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330417    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330467    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330508    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330553    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.330600    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\") pod \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\" (UID: \"6a33bb9e-1ddf-463a-ac22-d819b9c3181c\") "
Nov 02 15:30:57.341052 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.331482    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.342232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.315257    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:57.342478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.340991    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:57.342608 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.341136    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:57.345410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.345379    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:30:57.352458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.345715    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "kube-api-access-bhwhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:57.352458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.345945    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:30:57.352458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.346356    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:30:57.352458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.350309    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo" pod "6a33bb9e-1ddf-463a-ac22-d819b9c3181c" (UID: "6a33bb9e-1ddf-463a-ac22-d819b9c3181c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:30:57.432982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.432935    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.432982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.432971    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.432982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.432982    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.432982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.432992    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.433324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.433003    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhwhj\" (UniqueName: \"kubernetes.io/projected/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-kube-api-access-bhwhj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.433324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.433012    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.433324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.433021    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:57.433324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:57.433031    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a33bb9e-1ddf-463a-ac22-d819b9c3181c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:30:58.028589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:58.028499    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"
Nov 02 15:30:58.028589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:58.028510    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2" event={"ID":"6a33bb9e-1ddf-463a-ac22-d819b9c3181c","Type":"ContainerDied","Data":"b6d82a01efe2b104e79b08e784a74157233a5bc6454a2b6c5088f18e06e2cf54"}
Nov 02 15:30:58.028589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:58.028566    4470 scope.go:122] "RemoveContainer" containerID="6dfb19632a434e83340781cb0ab879c69f304877a361c913942e1beae859993f"
Nov 02 15:30:58.231218 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:58.221761    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:30:58.275252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:58.267757    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-303ba8-openshift-default-68d7456995-8zbn2"]
Nov 02 15:30:59.707069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:30:59.707033    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" path="/var/lib/kubelet/pods/6a33bb9e-1ddf-463a-ac22-d819b9c3181c/volumes"
Nov 02 15:31:01.655672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:01.655628    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" podUID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" containerName="payload-processing" containerID="cri-o://d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d" gracePeriod=25
Nov 02 15:31:01.766765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:01.766682    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" podUID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" containerName="payload-pre-processing" containerID="cri-o://4c3ab10674a616e4c6c3f979a6ebb9a25bcf60b67054c9b25f80340bb7d3a1de" gracePeriod=25
Nov 02 15:31:02.020250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.020108    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:31:02.065498 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.065456    4470 generic.go:356] "Generic (PLEG): container finished" podID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" containerID="4c3ab10674a616e4c6c3f979a6ebb9a25bcf60b67054c9b25f80340bb7d3a1de" exitCode=0
Nov 02 15:31:02.065672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.065580    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" event={"ID":"af4c4fe7-8695-4c60-aac9-b7acd1ef5432","Type":"ContainerDied","Data":"4c3ab10674a616e4c6c3f979a6ebb9a25bcf60b67054c9b25f80340bb7d3a1de"}
Nov 02 15:31:02.067471 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.067439    4470 generic.go:356] "Generic (PLEG): container finished" podID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" containerID="d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d" exitCode=0
Nov 02 15:31:02.067646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.067609    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" event={"ID":"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5","Type":"ContainerDied","Data":"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"}
Nov 02 15:31:02.067815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.067773    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss" event={"ID":"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5","Type":"ContainerDied","Data":"3ddd5445221f8cb8a0b040d7f1bd15a6e9484ffa65970b4cba7ecfb75cbf29a5"}
Nov 02 15:31:02.067884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.067843    4470 scope.go:122] "RemoveContainer" containerID="d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"
Nov 02 15:31:02.068225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.068143    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"
Nov 02 15:31:02.117868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.117836    4470 scope.go:122] "RemoveContainer" containerID="d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"
Nov 02 15:31:02.118290 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:02.118261    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d\": container with ID starting with d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d not found: ID does not exist" containerID="d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"
Nov 02 15:31:02.118407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.118304    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d"} err="failed to get container status \"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d\": rpc error: code = NotFound desc = could not find container \"d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d\": container with ID starting with d328da24ec789a505ac42ad8cc6f97ea7b7d3248ce841bf0300f8cd3d50f881d not found: ID does not exist"
Nov 02 15:31:02.134669 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.134637    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\" (UniqueName: \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\") pod \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") "
Nov 02 15:31:02.134843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.134680    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\") pod \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\" (UID: \"3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5\") "
Nov 02 15:31:02.135616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.135588    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume" pod "3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" (UID: "3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:02.137371 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.137342    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng" pod "3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" (UID: "3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5"). InnerVolumeSpecName "kube-api-access-9fwng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:02.312852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.245821    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fwng\" (UniqueName: \"kubernetes.io/projected/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-kube-api-access-9fwng\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:02.312852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.245860    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:02.312852 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.269453    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:31:02.390217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.383580    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\" (UniqueName: \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\") pod \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") "
Nov 02 15:31:02.390217 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.383861    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\") pod \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\" (UID: \"af4c4fe7-8695-4c60-aac9-b7acd1ef5432\") "
Nov 02 15:31:02.431211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.397613    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume" pod "af4c4fe7-8695-4c60-aac9-b7acd1ef5432" (UID: "af4c4fe7-8695-4c60-aac9-b7acd1ef5432"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:02.431211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.405497    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:02.459214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.447414    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc" pod "af4c4fe7-8695-4c60-aac9-b7acd1ef5432" (UID: "af4c4fe7-8695-4c60-aac9-b7acd1ef5432"). InnerVolumeSpecName "kube-api-access-ld4nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:02.539212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.538553    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld4nc\" (UniqueName: \"kubernetes.io/projected/af4c4fe7-8695-4c60-aac9-b7acd1ef5432-kube-api-access-ld4nc\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:02.758221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.740395    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:31:02.761654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:02.761437    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-468af4-6b5fb57cd7-wjkss"]
Nov 02 15:31:03.096536 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.096499    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q" event={"ID":"af4c4fe7-8695-4c60-aac9-b7acd1ef5432","Type":"ContainerDied","Data":"a5c5120984a0d109e1bedca9645949ed6a51e42720a79c0120487c4e5874ac64"}
Nov 02 15:31:03.096819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.096804    4470 scope.go:122] "RemoveContainer" containerID="4c3ab10674a616e4c6c3f979a6ebb9a25bcf60b67054c9b25f80340bb7d3a1de"
Nov 02 15:31:03.097009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.096997    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"
Nov 02 15:31:03.264487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.264448    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:31:03.297337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.297294    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-468af4-764f4c5dc9-r6d4q"]
Nov 02 15:31:03.567066 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:03.567028    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:31:03.709000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.708968    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" path="/var/lib/kubelet/pods/3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5/volumes"
Nov 02 15:31:03.709420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:03.709405    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" path="/var/lib/kubelet/pods/af4c4fe7-8695-4c60-aac9-b7acd1ef5432/volumes"
Nov 02 15:31:08.171312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.171177    4470 generic.go:356] "Generic (PLEG): container finished" podID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerID="c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f" exitCode=0
Nov 02 15:31:08.171312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.171247    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerDied","Data":"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"}
Nov 02 15:31:08.171862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.171646    4470 scope.go:122] "RemoveContainer" containerID="c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"
Nov 02 15:31:08.173255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.173221    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:31:08.173388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.173317    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:31:08.173388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:08.173367    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:31:09.200795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.200755    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerStarted","Data":"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"}
Nov 02 15:31:09.437824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.437788    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:31:09.596394 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.596291    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:31:09.596635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.596608    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" podUID="9979a001-a253-40e5-a602-8dd87cbd40b5" containerName="maas-api" containerID="cri-o://b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303" gracePeriod=30
Nov 02 15:31:09.696212 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.695618    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:31:09.698337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.698260    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:31:09.698483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:09.698361    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:31:10.162484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.161713    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:31:10.210527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.210488    4470 generic.go:356] "Generic (PLEG): container finished" podID="9979a001-a253-40e5-a602-8dd87cbd40b5" containerID="b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303" exitCode=0
Nov 02 15:31:10.210984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.210541    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"
Nov 02 15:31:10.210984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.210583    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" event={"ID":"9979a001-a253-40e5-a602-8dd87cbd40b5","Type":"ContainerDied","Data":"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"}
Nov 02 15:31:10.210984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.210645    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks" event={"ID":"9979a001-a253-40e5-a602-8dd87cbd40b5","Type":"ContainerDied","Data":"ff6fe51203764895eafc813cdf9f83212fb72deee24fc5ec9181a7557fe4ca1a"}
Nov 02 15:31:10.210984 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.210672    4470 scope.go:122] "RemoveContainer" containerID="b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"
Nov 02 15:31:10.224420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.224399    4470 scope.go:122] "RemoveContainer" containerID="b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"
Nov 02 15:31:10.224741 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:10.224717    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303\": container with ID starting with b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303 not found: ID does not exist" containerID="b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"
Nov 02 15:31:10.224786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.224753    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303"} err="failed to get container status \"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303\": rpc error: code = NotFound desc = could not find container \"b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303\": container with ID starting with b4639e6fed182671a5ef1a6035b2e656f82b3f184eee1f97dcad3e2fb2d36303 not found: ID does not exist"
Nov 02 15:31:10.291269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.289707    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\" (UniqueName: \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\") pod \"9979a001-a253-40e5-a602-8dd87cbd40b5\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") "
Nov 02 15:31:10.291269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.289805    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") pod \"9979a001-a253-40e5-a602-8dd87cbd40b5\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") "
Nov 02 15:31:10.291269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.290077    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\") pod \"9979a001-a253-40e5-a602-8dd87cbd40b5\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") "
Nov 02 15:31:10.291269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.290113    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\") pod \"9979a001-a253-40e5-a602-8dd87cbd40b5\" (UID: \"9979a001-a253-40e5-a602-8dd87cbd40b5\") "
Nov 02 15:31:10.299246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.297410    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca" pod "9979a001-a253-40e5-a602-8dd87cbd40b5" (UID: "9979a001-a253-40e5-a602-8dd87cbd40b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:10.303036 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.303004    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n" pod "9979a001-a253-40e5-a602-8dd87cbd40b5" (UID: "9979a001-a253-40e5-a602-8dd87cbd40b5"). InnerVolumeSpecName "kube-api-access-9nc8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:10.303898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.303860    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs" pod "9979a001-a253-40e5-a602-8dd87cbd40b5" (UID: "9979a001-a253-40e5-a602-8dd87cbd40b5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:31:10.304746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.304712    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls" pod "9979a001-a253-40e5-a602-8dd87cbd40b5" (UID: "9979a001-a253-40e5-a602-8dd87cbd40b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:31:10.397395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.397348    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9979a001-a253-40e5-a602-8dd87cbd40b5-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:10.397662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.397645    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:10.397773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.397762    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nc8n\" (UniqueName: \"kubernetes.io/projected/9979a001-a253-40e5-a602-8dd87cbd40b5-kube-api-access-9nc8n\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:10.397892 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.397879    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9979a001-a253-40e5-a602-8dd87cbd40b5-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:10.655176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.655058    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:31:10.710214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.698607    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:31:10.710214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.698671    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:31:10.710214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.702992    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-559973-54f65b7bdd-sh8ks"]
Nov 02 15:31:10.916418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.916332    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:31:10.916576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:10.916532    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" podUID="aab15205-83f1-4672-93c8-61fb8a184d33" containerName="maas-api" containerID="cri-o://e759eebb9a81da190e4338398f134565570d9315c8b368b497d1b2bae0427ac6" gracePeriod=30
Nov 02 15:31:11.256801 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.255723    4470 generic.go:356] "Generic (PLEG): container finished" podID="aab15205-83f1-4672-93c8-61fb8a184d33" containerID="e759eebb9a81da190e4338398f134565570d9315c8b368b497d1b2bae0427ac6" exitCode=0
Nov 02 15:31:11.256801 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.255827    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" event={"ID":"aab15205-83f1-4672-93c8-61fb8a184d33","Type":"ContainerDied","Data":"e759eebb9a81da190e4338398f134565570d9315c8b368b497d1b2bae0427ac6"}
Nov 02 15:31:11.535061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.534981    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:31:11.594259 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.593644    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\") pod \"aab15205-83f1-4672-93c8-61fb8a184d33\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") "
Nov 02 15:31:11.594473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.594388    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\" (UniqueName: \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\") pod \"aab15205-83f1-4672-93c8-61fb8a184d33\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") "
Nov 02 15:31:11.594473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.594432    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") pod \"aab15205-83f1-4672-93c8-61fb8a184d33\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") "
Nov 02 15:31:11.594589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.594546    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\") pod \"aab15205-83f1-4672-93c8-61fb8a184d33\" (UID: \"aab15205-83f1-4672-93c8-61fb8a184d33\") "
Nov 02 15:31:11.595544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.595510    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca" pod "aab15205-83f1-4672-93c8-61fb8a184d33" (UID: "aab15205-83f1-4672-93c8-61fb8a184d33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:11.597351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.597323    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2" pod "aab15205-83f1-4672-93c8-61fb8a184d33" (UID: "aab15205-83f1-4672-93c8-61fb8a184d33"). InnerVolumeSpecName "kube-api-access-7hdw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:11.597448 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.597345    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls" pod "aab15205-83f1-4672-93c8-61fb8a184d33" (UID: "aab15205-83f1-4672-93c8-61fb8a184d33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:31:11.597578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.597559    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs" pod "aab15205-83f1-4672-93c8-61fb8a184d33" (UID: "aab15205-83f1-4672-93c8-61fb8a184d33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:31:11.703638 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.703603    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:31:11.703899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.703870    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:31:11.704345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.704321    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aab15205-83f1-4672-93c8-61fb8a184d33-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:11.704445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.704345    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hdw2\" (UniqueName: \"kubernetes.io/projected/aab15205-83f1-4672-93c8-61fb8a184d33-kube-api-access-7hdw2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:11.704445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.704362    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:11.704445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.704375    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aab15205-83f1-4672-93c8-61fb8a184d33-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:11.714400 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:11.714369    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9979a001-a253-40e5-a602-8dd87cbd40b5" path="/var/lib/kubelet/pods/9979a001-a253-40e5-a602-8dd87cbd40b5/volumes"
Nov 02 15:31:12.269238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.266421    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"
Nov 02 15:31:12.269238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.266447    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m" event={"ID":"aab15205-83f1-4672-93c8-61fb8a184d33","Type":"ContainerDied","Data":"43463b5c97d7dfb354f9bf6d835145d741723b997f03a692925900f028bc5ee3"}
Nov 02 15:31:12.269238 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.266516    4470 scope.go:122] "RemoveContainer" containerID="e759eebb9a81da190e4338398f134565570d9315c8b368b497d1b2bae0427ac6"
Nov 02 15:31:12.437982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.437937    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:31:12.502071 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.502022    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-6b2f61-5ff4cf68d8-m9m2m"]
Nov 02 15:31:12.694676 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.694640    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:31:12.694855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:12.694715    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:31:13.695659 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:13.695622    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:31:13.696101 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:13.695679    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:31:13.715758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:13.715726    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aab15205-83f1-4672-93c8-61fb8a184d33" path="/var/lib/kubelet/pods/aab15205-83f1-4672-93c8-61fb8a184d33/volumes"
Nov 02 15:31:14.699515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:14.699479    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:31:14.700636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:14.700616    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:31:31.521465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:31.521427    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:31:31.617795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:31.617757    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:31:33.566565 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:33.566523    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:31:34.011119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:34.011088    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:31:34.043813 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:34.043787    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:31:36.521887 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.521812    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" podUID="28c27b68-8549-42d5-841d-3c396bcaa889" containerName="payload-processing" containerID="cri-o://5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8" gracePeriod=25
Nov 02 15:31:36.618119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.618073    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" podUID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" containerName="payload-pre-processing" containerID="cri-o://e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592" gracePeriod=25
Nov 02 15:31:36.904475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.904446    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:31:36.929291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.929256    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:31:36.953173 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953147    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\") pod \"28c27b68-8549-42d5-841d-3c396bcaa889\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") "
Nov 02 15:31:36.953331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953212    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\" (UniqueName: \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\") pod \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") "
Nov 02 15:31:36.953382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953334    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\" (UniqueName: \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\") pod \"28c27b68-8549-42d5-841d-3c396bcaa889\" (UID: \"28c27b68-8549-42d5-841d-3c396bcaa889\") "
Nov 02 15:31:36.953437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953409    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\") pod \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\" (UID: \"5b23d5ba-d054-4835-8f5a-e6886e76c1b3\") "
Nov 02 15:31:36.953657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953580    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume" pod "28c27b68-8549-42d5-841d-3c396bcaa889" (UID: "28c27b68-8549-42d5-841d-3c396bcaa889"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:36.953804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953779    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume" pod "5b23d5ba-d054-4835-8f5a-e6886e76c1b3" (UID: "5b23d5ba-d054-4835-8f5a-e6886e76c1b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:31:36.953886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.953870    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c27b68-8549-42d5-841d-3c396bcaa889-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:36.955344 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.955321    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l" pod "5b23d5ba-d054-4835-8f5a-e6886e76c1b3" (UID: "5b23d5ba-d054-4835-8f5a-e6886e76c1b3"). InnerVolumeSpecName "kube-api-access-4gx6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:36.955461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:36.955443    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl" pod "28c27b68-8549-42d5-841d-3c396bcaa889" (UID: "28c27b68-8549-42d5-841d-3c396bcaa889"). InnerVolumeSpecName "kube-api-access-rk7nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:31:37.055269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.055166    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:37.055269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.055211    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gx6l\" (UniqueName: \"kubernetes.io/projected/5b23d5ba-d054-4835-8f5a-e6886e76c1b3-kube-api-access-4gx6l\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:37.055269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.055225    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk7nl\" (UniqueName: \"kubernetes.io/projected/28c27b68-8549-42d5-841d-3c396bcaa889-kube-api-access-rk7nl\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:31:37.488779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.488744    4470 generic.go:356] "Generic (PLEG): container finished" podID="28c27b68-8549-42d5-841d-3c396bcaa889" containerID="5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8" exitCode=0
Nov 02 15:31:37.488983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.488794    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"
Nov 02 15:31:37.488983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.488825    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" event={"ID":"28c27b68-8549-42d5-841d-3c396bcaa889","Type":"ContainerDied","Data":"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"}
Nov 02 15:31:37.488983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.488880    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl" event={"ID":"28c27b68-8549-42d5-841d-3c396bcaa889","Type":"ContainerDied","Data":"19126366b123e6867c3e8c5e3c9a789f3a140fd754b9f5d8a661dcb858e8679f"}
Nov 02 15:31:37.488983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.488927    4470 scope.go:122] "RemoveContainer" containerID="5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"
Nov 02 15:31:37.490160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.490138    4470 generic.go:356] "Generic (PLEG): container finished" podID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" containerID="e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592" exitCode=0
Nov 02 15:31:37.490267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.490177    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"
Nov 02 15:31:37.490267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.490221    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" event={"ID":"5b23d5ba-d054-4835-8f5a-e6886e76c1b3","Type":"ContainerDied","Data":"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"}
Nov 02 15:31:37.490369 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.490270    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv" event={"ID":"5b23d5ba-d054-4835-8f5a-e6886e76c1b3","Type":"ContainerDied","Data":"a3b9f200c686b107e6d485e5dfd2c27596501772d9c129aeab4b912dbb7f4b95"}
Nov 02 15:31:37.500769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.500753    4470 scope.go:122] "RemoveContainer" containerID="5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"
Nov 02 15:31:37.501065 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:37.501043    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8\": container with ID starting with 5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8 not found: ID does not exist" containerID="5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"
Nov 02 15:31:37.501154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.501072    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8"} err="failed to get container status \"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8\": rpc error: code = NotFound desc = could not find container \"5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8\": container with ID starting with 5559453433c212819051568e6408efa10f154441d6680f605619f460681dffb8 not found: ID does not exist"
Nov 02 15:31:37.501154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.501093    4470 scope.go:122] "RemoveContainer" containerID="e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"
Nov 02 15:31:37.510158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.510141    4470 scope.go:122] "RemoveContainer" containerID="e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"
Nov 02 15:31:37.510487 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:31:37.510466    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592\": container with ID starting with e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592 not found: ID does not exist" containerID="e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"
Nov 02 15:31:37.510533 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.510494    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592"} err="failed to get container status \"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592\": rpc error: code = NotFound desc = could not find container \"e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592\": container with ID starting with e012aeea6e007b4c1eda1ff26a553baf1633abc57d79c013a12476b8cfa63592 not found: ID does not exist"
Nov 02 15:31:37.654772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.654738    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:31:37.657902 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.657874    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-d6b9c6-658546944c-lnckv"]
Nov 02 15:31:37.707784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.707753    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" path="/var/lib/kubelet/pods/5b23d5ba-d054-4835-8f5a-e6886e76c1b3/volumes"
Nov 02 15:31:37.765713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.765608    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:31:37.770059 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:37.770033    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-d6b9c6-668d9d44b7-q7wsl"]
Nov 02 15:31:39.708800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:31:39.708762    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28c27b68-8549-42d5-841d-3c396bcaa889" path="/var/lib/kubelet/pods/28c27b68-8549-42d5-841d-3c396bcaa889/volumes"
Nov 02 15:32:03.566725 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:03.566678    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:32:04.659356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:04.659317    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:32:04.659781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:04.659542    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" podUID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" containerName="maas-api" containerID="cri-o://43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf" gracePeriod=30
Nov 02 15:32:04.986074 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:04.986037    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:32:05.064210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.062644    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\") pod \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") "
Nov 02 15:32:05.064210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.062749    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\") pod \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") "
Nov 02 15:32:05.064210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.062903    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") pod \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") "
Nov 02 15:32:05.064210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.063001    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\" (UniqueName: \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\") pod \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\" (UID: \"ce766b1b-5caa-44b2-8209-55baa7cb41e6\") "
Nov 02 15:32:05.069645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.065550    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca" pod "ce766b1b-5caa-44b2-8209-55baa7cb41e6" (UID: "ce766b1b-5caa-44b2-8209-55baa7cb41e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:05.087567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.087363    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls" pod "ce766b1b-5caa-44b2-8209-55baa7cb41e6" (UID: "ce766b1b-5caa-44b2-8209-55baa7cb41e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:05.087567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.087484    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs" pod "ce766b1b-5caa-44b2-8209-55baa7cb41e6" (UID: "ce766b1b-5caa-44b2-8209-55baa7cb41e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:05.099765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.094461    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk" pod "ce766b1b-5caa-44b2-8209-55baa7cb41e6" (UID: "ce766b1b-5caa-44b2-8209-55baa7cb41e6"). InnerVolumeSpecName "kube-api-access-4g6qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:05.165015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.164960    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce766b1b-5caa-44b2-8209-55baa7cb41e6-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:05.165015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.165015    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:05.165015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.165031    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce766b1b-5caa-44b2-8209-55baa7cb41e6-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:05.165360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.165045    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g6qk\" (UniqueName: \"kubernetes.io/projected/ce766b1b-5caa-44b2-8209-55baa7cb41e6-kube-api-access-4g6qk\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:05.685886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.685851    4470 generic.go:356] "Generic (PLEG): container finished" podID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" containerID="43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf" exitCode=0
Nov 02 15:32:05.686373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.685906    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"
Nov 02 15:32:05.686373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.685909    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" event={"ID":"ce766b1b-5caa-44b2-8209-55baa7cb41e6","Type":"ContainerDied","Data":"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"}
Nov 02 15:32:05.686373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.686033    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x" event={"ID":"ce766b1b-5caa-44b2-8209-55baa7cb41e6","Type":"ContainerDied","Data":"0b4d1dd1332bc61d447d08bb67180c2fe53aa18c035eb4935b8e931d53cf6d22"}
Nov 02 15:32:05.686373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.686055    4470 scope.go:122] "RemoveContainer" containerID="43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"
Nov 02 15:32:05.697730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.697706    4470 scope.go:122] "RemoveContainer" containerID="43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"
Nov 02 15:32:05.698038 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:05.698018    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf\": container with ID starting with 43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf not found: ID does not exist" containerID="43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"
Nov 02 15:32:05.698095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.698050    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf"} err="failed to get container status \"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf\": rpc error: code = NotFound desc = could not find container \"43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf\": container with ID starting with 43901ce757ce0919ee18682ec22cdd03949d2db61ca2ba87d05ef04a02ed42cf not found: ID does not exist"
Nov 02 15:32:05.852123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.852084    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:32:05.901753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:05.901723    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-468af4-687d6cd55-26v5x"]
Nov 02 15:32:07.706582 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:07.706547    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" path="/var/lib/kubelet/pods/ce766b1b-5caa-44b2-8209-55baa7cb41e6/volumes"
Nov 02 15:32:21.691806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:21.691773    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:32:21.692255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:21.692022    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" podUID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" containerName="maas-api" containerID="cri-o://1d18cc607de4ffe7c1905059467b1aba52b598f1ce3ba461010de3d66776ee00" gracePeriod=30
Nov 02 15:32:21.865834 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:21.865797    4470 generic.go:356] "Generic (PLEG): container finished" podID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" containerID="1d18cc607de4ffe7c1905059467b1aba52b598f1ce3ba461010de3d66776ee00" exitCode=0
Nov 02 15:32:21.866010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:21.865866    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" event={"ID":"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11","Type":"ContainerDied","Data":"1d18cc607de4ffe7c1905059467b1aba52b598f1ce3ba461010de3d66776ee00"}
Nov 02 15:32:22.018095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.018066    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:32:22.144526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.144494    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\" (UniqueName: \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\") pod \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") "
Nov 02 15:32:22.144526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.144535    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\") pod \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") "
Nov 02 15:32:22.144749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.144718    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\") pod \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") "
Nov 02 15:32:22.144915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.144896    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\") pod \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\" (UID: \"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11\") "
Nov 02 15:32:22.145227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.145202    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca" pod "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" (UID: "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:22.145622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.145602    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:22.146855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.146826    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs" pod "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" (UID: "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:22.147347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.147322    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls" pod "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" (UID: "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:22.147509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.147489    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85" pod "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" (UID: "66757923-f4f5-4cde-9dc8-a9b0f3f1dc11"). InnerVolumeSpecName "kube-api-access-spr85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:22.246843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.246754    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:22.246843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.246788    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spr85\" (UniqueName: \"kubernetes.io/projected/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-kube-api-access-spr85\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:22.246843 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.246805    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:22.875232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.875182    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"
Nov 02 15:32:22.875688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.875238    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2" event={"ID":"66757923-f4f5-4cde-9dc8-a9b0f3f1dc11","Type":"ContainerDied","Data":"db8249b2f021dba6fc0a690a555df483c60d30a2ddb25e4495c907f2578b14e8"}
Nov 02 15:32:22.875688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:22.875300    4470 scope.go:122] "RemoveContainer" containerID="1d18cc607de4ffe7c1905059467b1aba52b598f1ce3ba461010de3d66776ee00"
Nov 02 15:32:23.041107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:23.041072    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:32:23.047121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:23.047089    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-bcsq2"]
Nov 02 15:32:23.779219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:23.778134    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" path="/var/lib/kubelet/pods/66757923-f4f5-4cde-9dc8-a9b0f3f1dc11/volumes"
Nov 02 15:32:24.540304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:24.540267    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:32:24.541310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:24.541273    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" containerID="cri-o://21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3" gracePeriod=30
Nov 02 15:32:27.874337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:27.874296    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:32:27.874878 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:27.874519    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" podUID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" containerName="maas-api" containerID="cri-o://6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504" gracePeriod=30
Nov 02 15:32:28.194282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.194250    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:32:28.351409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.350147    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") pod \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") "
Nov 02 15:32:28.351409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.350417    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\" (UniqueName: \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\") pod \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") "
Nov 02 15:32:28.351409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.350466    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\") pod \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") "
Nov 02 15:32:28.351409 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.350555    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\") pod \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\" (UID: \"5b22ca92-967c-4aef-94af-7ce2f7fd017d\") "
Nov 02 15:32:28.352255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.351914    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca" pod "5b22ca92-967c-4aef-94af-7ce2f7fd017d" (UID: "5b22ca92-967c-4aef-94af-7ce2f7fd017d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:28.376123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.375136    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls" pod "5b22ca92-967c-4aef-94af-7ce2f7fd017d" (UID: "5b22ca92-967c-4aef-94af-7ce2f7fd017d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:28.376123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.376005    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs" pod "5b22ca92-967c-4aef-94af-7ce2f7fd017d" (UID: "5b22ca92-967c-4aef-94af-7ce2f7fd017d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:28.377228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.377180    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl" pod "5b22ca92-967c-4aef-94af-7ce2f7fd017d" (UID: "5b22ca92-967c-4aef-94af-7ce2f7fd017d"). InnerVolumeSpecName "kube-api-access-7hdjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:28.465500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.465444    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hdjl\" (UniqueName: \"kubernetes.io/projected/5b22ca92-967c-4aef-94af-7ce2f7fd017d-kube-api-access-7hdjl\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:28.465821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.465794    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:28.465821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.465823    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b22ca92-967c-4aef-94af-7ce2f7fd017d-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:28.466095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.465851    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b22ca92-967c-4aef-94af-7ce2f7fd017d-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:28.974010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.973973    4470 generic.go:356] "Generic (PLEG): container finished" podID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" containerID="6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504" exitCode=0
Nov 02 15:32:28.974791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.974342    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"
Nov 02 15:32:28.978501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.974364    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" event={"ID":"5b22ca92-967c-4aef-94af-7ce2f7fd017d","Type":"ContainerDied","Data":"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"}
Nov 02 15:32:28.978501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.978323    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l" event={"ID":"5b22ca92-967c-4aef-94af-7ce2f7fd017d","Type":"ContainerDied","Data":"797102c40c0c687264be1d74674bcbc97d152995ac4fa40b910d8c44b4291862"}
Nov 02 15:32:28.978501 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:28.978360    4470 scope.go:122] "RemoveContainer" containerID="6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"
Nov 02 15:32:29.007089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.006170    4470 scope.go:122] "RemoveContainer" containerID="6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"
Nov 02 15:32:29.007089 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:29.006556    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504\": container with ID starting with 6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504 not found: ID does not exist" containerID="6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"
Nov 02 15:32:29.007089 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.006590    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504"} err="failed to get container status \"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504\": rpc error: code = NotFound desc = could not find container \"6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504\": container with ID starting with 6b0f4bbffaa13d65a3efef90b09dc40158c214449f5608f4cf0bc66220311504 not found: ID does not exist"
Nov 02 15:32:29.153121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.153085    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:32:29.158136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.158110    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d6b9c6-564d5c666f-b762l"]
Nov 02 15:32:29.227732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.227651    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:32:29.227900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.227864    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" podUID="970598d6-a53d-4128-9043-8af35fead769" containerName="maas-api" containerID="cri-o://eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66" gracePeriod=30
Nov 02 15:32:29.533419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.533395    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:32:29.590285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.590258    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\") pod \"970598d6-a53d-4128-9043-8af35fead769\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") "
Nov 02 15:32:29.590436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.590363    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") pod \"970598d6-a53d-4128-9043-8af35fead769\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") "
Nov 02 15:32:29.590436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.590417    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\" (UniqueName: \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\") pod \"970598d6-a53d-4128-9043-8af35fead769\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") "
Nov 02 15:32:29.590558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.590469    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\") pod \"970598d6-a53d-4128-9043-8af35fead769\" (UID: \"970598d6-a53d-4128-9043-8af35fead769\") "
Nov 02 15:32:29.590916 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.590882    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca" pod "970598d6-a53d-4128-9043-8af35fead769" (UID: "970598d6-a53d-4128-9043-8af35fead769"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:29.592345 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.592322    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls" pod "970598d6-a53d-4128-9043-8af35fead769" (UID: "970598d6-a53d-4128-9043-8af35fead769"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:29.592815 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.592795    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs" pod "970598d6-a53d-4128-9043-8af35fead769" (UID: "970598d6-a53d-4128-9043-8af35fead769"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:29.592884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.592802    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2" pod "970598d6-a53d-4128-9043-8af35fead769" (UID: "970598d6-a53d-4128-9043-8af35fead769"). InnerVolumeSpecName "kube-api-access-pkzf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:29.698262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.692404    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:29.698262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.692441    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkzf2\" (UniqueName: \"kubernetes.io/projected/970598d6-a53d-4128-9043-8af35fead769-kube-api-access-pkzf2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:29.698262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.692457    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/970598d6-a53d-4128-9043-8af35fead769-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:29.698262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.692471    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/970598d6-a53d-4128-9043-8af35fead769-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:29.722347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.721336    4470 patch_prober.go:29] interesting pod/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused" start-of-body=
Nov 02 15:32:29.722889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.722602    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.128:15021/healthz/ready\": dial tcp 10.133.0.128:15021: connect: connection refused"
Nov 02 15:32:29.763303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.763214    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" path="/var/lib/kubelet/pods/5b22ca92-967c-4aef-94af-7ce2f7fd017d/volumes"
Nov 02 15:32:29.942586 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.942557    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:32:29.984048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.984013    4470 generic.go:356] "Generic (PLEG): container finished" podID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerID="21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3" exitCode=0
Nov 02 15:32:29.984526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.984057    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"
Nov 02 15:32:29.984526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.984091    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerDied","Data":"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"}
Nov 02 15:32:29.984526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.984144    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz" event={"ID":"93c7016c-1dc1-4269-9a32-ed3c1c264e1a","Type":"ContainerDied","Data":"f95b525d170bc79deccdfa00c7f4a5944c61ed951ac15bdd1cd134617e26a322"}
Nov 02 15:32:29.984526 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.984171    4470 scope.go:122] "RemoveContainer" containerID="21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"
Nov 02 15:32:29.985751 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.985732    4470 generic.go:356] "Generic (PLEG): container finished" podID="970598d6-a53d-4128-9043-8af35fead769" containerID="eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66" exitCode=0
Nov 02 15:32:29.985873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.985768    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" event={"ID":"970598d6-a53d-4128-9043-8af35fead769","Type":"ContainerDied","Data":"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"}
Nov 02 15:32:29.985873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.985779    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"
Nov 02 15:32:29.985873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.985797    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm" event={"ID":"970598d6-a53d-4128-9043-8af35fead769","Type":"ContainerDied","Data":"1f7a3adf0fc6d0172c4118dc530e1967236fedcbe1ca30ace583bd880c8f9dec"}
Nov 02 15:32:29.994779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:29.994760    4470 scope.go:122] "RemoveContainer" containerID="c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"
Nov 02 15:32:30.004329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.004310    4470 scope.go:122] "RemoveContainer" containerID="21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"
Nov 02 15:32:30.004587 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:30.004567    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3\": container with ID starting with 21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3 not found: ID does not exist" containerID="21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"
Nov 02 15:32:30.004644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.004600    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3"} err="failed to get container status \"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3\": rpc error: code = NotFound desc = could not find container \"21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3\": container with ID starting with 21d7499c6ec07bc61ebe1a634581a97420883c150928bfe8fa111e6f2e1d94f3 not found: ID does not exist"
Nov 02 15:32:30.004644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.004621    4470 scope.go:122] "RemoveContainer" containerID="c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"
Nov 02 15:32:30.004874 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:30.004857    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f\": container with ID starting with c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f not found: ID does not exist" containerID="c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"
Nov 02 15:32:30.004928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.004880    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f"} err="failed to get container status \"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f\": rpc error: code = NotFound desc = could not find container \"c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f\": container with ID starting with c23a31bfe276f5c410ed08adb70cb49e1f0d2332faf28fdebb71727c86584c1f not found: ID does not exist"
Nov 02 15:32:30.004928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.004894    4470 scope.go:122] "RemoveContainer" containerID="eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"
Nov 02 15:32:30.013395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.013354    4470 scope.go:122] "RemoveContainer" containerID="eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"
Nov 02 15:32:30.013630 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:30.013608    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66\": container with ID starting with eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66 not found: ID does not exist" containerID="eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"
Nov 02 15:32:30.013679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.013637    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66"} err="failed to get container status \"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66\": rpc error: code = NotFound desc = could not find container \"eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66\": container with ID starting with eb6a37bd5f854114411c8eb84fdc4fc4574becde45d074e5058d2682b71c6d66 not found: ID does not exist"
Nov 02 15:32:30.099056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099027    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099063    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099111    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099286    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099354    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099360    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:30.099598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099444    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099495    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:30.099598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099549    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099612    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:30.099766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099650    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099690    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\") pod \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\" (UID: \"93c7016c-1dc1-4269-9a32-ed3c1c264e1a\") "
Nov 02 15:32:30.099922 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099756    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:30.100005 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.099972    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:30.100249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.100229    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.100249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.100248    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.100417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.100257    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.100417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.100267    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.100417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.100275    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.102062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.102033    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:32:30.102202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.102167    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:30.102284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.102248    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "kube-api-access-mnf7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:30.102500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.102483    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy" pod "93c7016c-1dc1-4269-9a32-ed3c1c264e1a" (UID: "93c7016c-1dc1-4269-9a32-ed3c1c264e1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:30.175249 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.175205    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:32:30.180417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.180389    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-885c69-7868d79745-h5nmm"]
Nov 02 15:32:30.202016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.201988    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.202016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.202017    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnf7r\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-kube-api-access-mnf7r\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.202244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.202031    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.202244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.202045    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93c7016c-1dc1-4269-9a32-ed3c1c264e1a-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:30.445300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.445262    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:32:30.448040 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:30.448017    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-468af4-openshift-default-dd578d549-dbppz"]
Nov 02 15:32:31.767211 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:31.766551    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" path="/var/lib/kubelet/pods/93c7016c-1dc1-4269-9a32-ed3c1c264e1a/volumes"
Nov 02 15:32:31.767711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:31.767270    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="970598d6-a53d-4128-9043-8af35fead769" path="/var/lib/kubelet/pods/970598d6-a53d-4128-9043-8af35fead769/volumes"
Nov 02 15:32:33.566845 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:33.566796    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:32:37.498731 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:37.498674    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:32:37.499335 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:37.499008    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy" containerID="cri-o://868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d" gracePeriod=30
Nov 02 15:32:39.277749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:39.277711    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:32:39.278213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:39.277952    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy" containerID="cri-o://fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5" gracePeriod=30
Nov 02 15:32:42.773209 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.773127    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:32:42.853635 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853600    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.853824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853663    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.853824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853721    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.853824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853813    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853853    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853916    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853971    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.853977    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854017    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854029    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854097    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\") pod \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\" (UID: \"ae98f0a5-0b8b-454c-8f16-d4a0f949277a\") "
Nov 02 15:32:42.854593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854396    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:42.855231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854710    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.855231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854733    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.855231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854748    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.855231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854881    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:42.855231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.854901    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:42.856368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.856346    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:42.856641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.856615    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "kube-api-access-2z7xh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:42.856757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.856729    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:32:42.857407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.857383    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token" pod "ae98f0a5-0b8b-454c-8f16-d4a0f949277a" (UID: "ae98f0a5-0b8b-454c-8f16-d4a0f949277a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:42.956246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956207    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.956246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956248    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.956488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956263    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.956488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956276    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.956488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956288    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:42.956488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:42.956301    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z7xh\" (UniqueName: \"kubernetes.io/projected/ae98f0a5-0b8b-454c-8f16-d4a0f949277a-kube-api-access-2z7xh\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:43.086844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.086750    4470 generic.go:356] "Generic (PLEG): container finished" podID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerID="868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d" exitCode=0
Nov 02 15:32:43.086844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.086812    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"
Nov 02 15:32:43.087034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.086834    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerDied","Data":"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"}
Nov 02 15:32:43.087034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.086913    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p" event={"ID":"ae98f0a5-0b8b-454c-8f16-d4a0f949277a","Type":"ContainerDied","Data":"7fcaaf1c26df9cda91c2bfe55401802513aef89c9c1d1ce90b1251f18dd4f4d4"}
Nov 02 15:32:43.087034 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.086940    4470 scope.go:122] "RemoveContainer" containerID="868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"
Nov 02 15:32:43.098107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.098080    4470 scope.go:122] "RemoveContainer" containerID="3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"
Nov 02 15:32:43.107769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.107740    4470 scope.go:122] "RemoveContainer" containerID="868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"
Nov 02 15:32:43.108108 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:43.108087    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d\": container with ID starting with 868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d not found: ID does not exist" containerID="868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"
Nov 02 15:32:43.108163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.108120    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d"} err="failed to get container status \"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d\": rpc error: code = NotFound desc = could not find container \"868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d\": container with ID starting with 868bfa48c724f3b48ec9959449877e7c4b15a8cac791be45c834490a4705113d not found: ID does not exist"
Nov 02 15:32:43.108163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.108139    4470 scope.go:122] "RemoveContainer" containerID="3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"
Nov 02 15:32:43.108528 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:43.108506    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550\": container with ID starting with 3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550 not found: ID does not exist" containerID="3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"
Nov 02 15:32:43.108567 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.108538    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550"} err="failed to get container status \"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550\": rpc error: code = NotFound desc = could not find container \"3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550\": container with ID starting with 3153c4c2c928dff22c23e305a20a25b94bc19666ed8a6a1edf9d7b8243e27550 not found: ID does not exist"
Nov 02 15:32:43.252899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.252859    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:32:43.257012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.256978    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d6b9c6-openshift-default-88bd5d4f-jdc2p"]
Nov 02 15:32:43.708027 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:43.707991    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" path="/var/lib/kubelet/pods/ae98f0a5-0b8b-454c-8f16-d4a0f949277a/volumes"
Nov 02 15:32:44.604979 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.604951    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:32:44.709116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709084    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709128    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709211    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709245    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709263    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709350    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709372    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709418    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709475    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:44.709643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709512    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\") pod \"6916f781-7088-43a3-811c-ab761ae73391\" (UID: \"6916f781-7088-43a3-811c-ab761ae73391\") "
Nov 02 15:32:44.709874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709659    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:44.709874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709792    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:44.709874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709866    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:44.710040 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.709907    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:44.710163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.710127    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.710163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.710154    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.710353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.710171    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.710353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.710202    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6916f781-7088-43a3-811c-ab761ae73391-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.710353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.710216    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.711537 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.711509    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:44.711642 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.711618    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:44.711887 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.711866    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:32:44.712018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.711998    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf" pod "6916f781-7088-43a3-811c-ab761ae73391" (UID: "6916f781-7088-43a3-811c-ab761ae73391"). InnerVolumeSpecName "kube-api-access-b9drf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:44.811898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.811864    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6916f781-7088-43a3-811c-ab761ae73391-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.811898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.811895    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6916f781-7088-43a3-811c-ab761ae73391-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.811898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.811908    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:44.812143 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:44.811917    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9drf\" (UniqueName: \"kubernetes.io/projected/6916f781-7088-43a3-811c-ab761ae73391-kube-api-access-b9drf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:45.102470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.102430    4470 generic.go:356] "Generic (PLEG): container finished" podID="6916f781-7088-43a3-811c-ab761ae73391" containerID="fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5" exitCode=0
Nov 02 15:32:45.102663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.102491    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"
Nov 02 15:32:45.102663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.102493    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerDied","Data":"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"}
Nov 02 15:32:45.102663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.102551    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg" event={"ID":"6916f781-7088-43a3-811c-ab761ae73391","Type":"ContainerDied","Data":"3f59860629ce1288ca4b2d62c80d349be425d11b7bb718fedd0efbf4f0e445ab"}
Nov 02 15:32:45.102663 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.102584    4470 scope.go:122] "RemoveContainer" containerID="fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"
Nov 02 15:32:45.113662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.113637    4470 scope.go:122] "RemoveContainer" containerID="fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"
Nov 02 15:32:45.123437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.123416    4470 scope.go:122] "RemoveContainer" containerID="fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"
Nov 02 15:32:45.123837 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:45.123816    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5\": container with ID starting with fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5 not found: ID does not exist" containerID="fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"
Nov 02 15:32:45.123913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.123845    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5"} err="failed to get container status \"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5\": rpc error: code = NotFound desc = could not find container \"fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5\": container with ID starting with fc84c74b7d2d1ac5bbd05fd2170350ceee034f1eb98dffae90990c12740beff5 not found: ID does not exist"
Nov 02 15:32:45.123913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.123864    4470 scope.go:122] "RemoveContainer" containerID="fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"
Nov 02 15:32:45.124135 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:45.124114    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142\": container with ID starting with fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142 not found: ID does not exist" containerID="fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"
Nov 02 15:32:45.124198 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.124141    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142"} err="failed to get container status \"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142\": rpc error: code = NotFound desc = could not find container \"fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142\": container with ID starting with fe09b42597cc24869ff45716905388bf59b71c6219843f137299bef71e033142 not found: ID does not exist"
Nov 02 15:32:45.268735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.268698    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:32:45.317857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.317823    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-885c69-openshift-default-85b7c65d57-4z2kg"]
Nov 02 15:32:45.707566 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:45.707535    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6916f781-7088-43a3-811c-ab761ae73391" path="/var/lib/kubelet/pods/6916f781-7088-43a3-811c-ab761ae73391/volumes"
Nov 02 15:32:47.619899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:47.619860    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:32:47.620406 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:47.620087    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" podUID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" containerName="maas-api" containerID="cri-o://0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b" gracePeriod=30
Nov 02 15:32:47.939401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:47.939362    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:32:48.073214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.072588    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\") pod \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") "
Nov 02 15:32:48.073214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.072647    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\" (UniqueName: \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\") pod \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") "
Nov 02 15:32:48.073214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.072758    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") pod \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") "
Nov 02 15:32:48.073214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.072819    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\") pod \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\" (UID: \"7ac4b109-4242-4ef3-bbe1-ca757eb493d5\") "
Nov 02 15:32:48.087235 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.078655    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca" pod "7ac4b109-4242-4ef3-bbe1-ca757eb493d5" (UID: "7ac4b109-4242-4ef3-bbe1-ca757eb493d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:48.090881 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.089351    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs" pod "7ac4b109-4242-4ef3-bbe1-ca757eb493d5" (UID: "7ac4b109-4242-4ef3-bbe1-ca757eb493d5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:48.097806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.096422    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls" pod "7ac4b109-4242-4ef3-bbe1-ca757eb493d5" (UID: "7ac4b109-4242-4ef3-bbe1-ca757eb493d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:32:48.098003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.097925    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz" pod "7ac4b109-4242-4ef3-bbe1-ca757eb493d5" (UID: "7ac4b109-4242-4ef3-bbe1-ca757eb493d5"). InnerVolumeSpecName "kube-api-access-kt2dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:48.135867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.135833    4470 generic.go:356] "Generic (PLEG): container finished" podID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" containerID="0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b" exitCode=0
Nov 02 15:32:48.136026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.135884    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"
Nov 02 15:32:48.136026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.135891    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" event={"ID":"7ac4b109-4242-4ef3-bbe1-ca757eb493d5","Type":"ContainerDied","Data":"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"}
Nov 02 15:32:48.136026 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.136012    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp" event={"ID":"7ac4b109-4242-4ef3-bbe1-ca757eb493d5","Type":"ContainerDied","Data":"f2f3d4b8639f287e3158ad6f4744f1284f73ac8834b2b60665eed0a30c6537c2"}
Nov 02 15:32:48.136171 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.136035    4470 scope.go:122] "RemoveContainer" containerID="0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"
Nov 02 15:32:48.146988 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.146961    4470 scope.go:122] "RemoveContainer" containerID="0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"
Nov 02 15:32:48.147435 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:32:48.147409    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b\": container with ID starting with 0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b not found: ID does not exist" containerID="0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"
Nov 02 15:32:48.147522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.147442    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b"} err="failed to get container status \"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b\": rpc error: code = NotFound desc = could not find container \"0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b\": container with ID starting with 0dd5601fea66155d5e1ad92b2041bcd4a8d7515af12a7d85122ddd9e0a633d6b not found: ID does not exist"
Nov 02 15:32:48.174626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.174538    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:48.174626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.174566    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt2dz\" (UniqueName: \"kubernetes.io/projected/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-kube-api-access-kt2dz\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:48.174626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.174583    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:48.174626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.174595    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac4b109-4242-4ef3-bbe1-ca757eb493d5-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:48.303046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.303008    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:32:48.307655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:48.307624    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-6dc69b-6d8ffb6f4f-9k2xp"]
Nov 02 15:32:49.716210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:49.713589    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" path="/var/lib/kubelet/pods/7ac4b109-4242-4ef3-bbe1-ca757eb493d5/volumes"
Nov 02 15:32:51.102360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:51.102323    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:32:51.102791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:51.102565    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy" containerID="cri-o://1e09c0e72cbfcbb03dcd4959151f67666cdab218347c8a9adaeafaabecdd75df" gracePeriod=30
Nov 02 15:32:56.246657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.246625    4470 generic.go:356] "Generic (PLEG): container finished" podID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerID="1e09c0e72cbfcbb03dcd4959151f67666cdab218347c8a9adaeafaabecdd75df" exitCode=0
Nov 02 15:32:56.247133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.246668    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerDied","Data":"1e09c0e72cbfcbb03dcd4959151f67666cdab218347c8a9adaeafaabecdd75df"}
Nov 02 15:32:56.247133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.246724    4470 scope.go:122] "RemoveContainer" containerID="aa2c3012d48e234f7d157fc0b042c19f5bdd9654a4c602d9b748c366ce644d22"
Nov 02 15:32:56.370812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.370746    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:32:56.494986 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.494947    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495007    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495075    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495340 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495229    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495310    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:56.495402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495340    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495402 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495382    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:56.495577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495432    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495492    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495531    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:56.495577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495552    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495599    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\") pod \"c780baf4-d4aa-4f00-b000-589f3b6a197f\" (UID: \"c780baf4-d4aa-4f00-b000-589f3b6a197f\") "
Nov 02 15:32:56.495966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495921    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:56.495966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.495845    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:32:56.496181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.496167    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c780baf4-d4aa-4f00-b000-589f3b6a197f-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.496271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.496224    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.496271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.496243    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.496271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.496256    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.496454 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.496270    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.497565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.497542    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:32:56.497706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.497687    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:32:56.497905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.497887    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:56.498014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.497991    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84" pod "c780baf4-d4aa-4f00-b000-589f3b6a197f" (UID: "c780baf4-d4aa-4f00-b000-589f3b6a197f"). InnerVolumeSpecName "kube-api-access-rmp84". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:32:56.597869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.597831    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.597869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.597866    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.597869 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.597877    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmp84\" (UniqueName: \"kubernetes.io/projected/c780baf4-d4aa-4f00-b000-589f3b6a197f-kube-api-access-rmp84\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:56.598118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:56.597890    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c780baf4-d4aa-4f00-b000-589f3b6a197f-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:32:57.256615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.256582    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm" event={"ID":"c780baf4-d4aa-4f00-b000-589f3b6a197f","Type":"ContainerDied","Data":"8a5dedd302a5f51e7993b3c79a6d8b1929cfa231516e1b5e8061ebe9d0b2808b"}
Nov 02 15:32:57.256615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.256603    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"
Nov 02 15:32:57.257165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.256642    4470 scope.go:122] "RemoveContainer" containerID="1e09c0e72cbfcbb03dcd4959151f67666cdab218347c8a9adaeafaabecdd75df"
Nov 02 15:32:57.471653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.471615    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:32:57.477053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.477023    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-b87e90-openshift-default-74bbfbb7b7-bdhfm"]
Nov 02 15:32:57.708738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:32:57.708701    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" path="/var/lib/kubelet/pods/c780baf4-d4aa-4f00-b000-589f3b6a197f/volumes"
Nov 02 15:33:03.428213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:03.428167    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:33:03.540728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:03.540695    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:33:03.567338 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:03.567305    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:33:08.428672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.428606    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" podUID="c1d28652-3d16-49be-8051-4a29139033e1" containerName="payload-processing" containerID="cri-o://c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a" gracePeriod=25
Nov 02 15:33:08.541458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.541414    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" podUID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" containerName="payload-pre-processing" containerID="cri-o://6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16" gracePeriod=25
Nov 02 15:33:08.700767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.700733    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:33:08.769349 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.769313    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\") pod \"c1d28652-3d16-49be-8051-4a29139033e1\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") "
Nov 02 15:33:08.769554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.769445    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\" (UniqueName: \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\") pod \"c1d28652-3d16-49be-8051-4a29139033e1\" (UID: \"c1d28652-3d16-49be-8051-4a29139033e1\") "
Nov 02 15:33:08.769733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.769707    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume" pod "c1d28652-3d16-49be-8051-4a29139033e1" (UID: "c1d28652-3d16-49be-8051-4a29139033e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:33:08.769865 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.769847    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d28652-3d16-49be-8051-4a29139033e1-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:08.772232 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.771997    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c" pod "c1d28652-3d16-49be-8051-4a29139033e1" (UID: "c1d28652-3d16-49be-8051-4a29139033e1"). InnerVolumeSpecName "kube-api-access-fhz7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:33:08.820290 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.820258    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:33:08.871221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.871153    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\" (UniqueName: \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\") pod \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") "
Nov 02 15:33:08.871376 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.871354    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\") pod \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\" (UID: \"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6\") "
Nov 02 15:33:08.871685 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.871667    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhz7c\" (UniqueName: \"kubernetes.io/projected/c1d28652-3d16-49be-8051-4a29139033e1-kube-api-access-fhz7c\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:08.871844 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.871704    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume" pod "ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" (UID: "ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:33:08.873383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.873364    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv" pod "ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" (UID: "ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6"). InnerVolumeSpecName "kube-api-access-c79hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:33:08.973244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.973113    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:08.973244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:08.973155    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c79hv\" (UniqueName: \"kubernetes.io/projected/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6-kube-api-access-c79hv\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:09.340836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.340733    4470 generic.go:356] "Generic (PLEG): container finished" podID="c1d28652-3d16-49be-8051-4a29139033e1" containerID="c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a" exitCode=0
Nov 02 15:33:09.340836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.340784    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"
Nov 02 15:33:09.340836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.340820    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" event={"ID":"c1d28652-3d16-49be-8051-4a29139033e1","Type":"ContainerDied","Data":"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"}
Nov 02 15:33:09.341145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.340918    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt" event={"ID":"c1d28652-3d16-49be-8051-4a29139033e1","Type":"ContainerDied","Data":"5a6b73fa9a22f51eec6d97f9716a253c646e4248fafa466b061f1644abbce9e2"}
Nov 02 15:33:09.341145 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.340952    4470 scope.go:122] "RemoveContainer" containerID="c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"
Nov 02 15:33:09.342368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.342339    4470 generic.go:356] "Generic (PLEG): container finished" podID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" containerID="6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16" exitCode=0
Nov 02 15:33:09.342515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.342397    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"
Nov 02 15:33:09.342515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.342422    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" event={"ID":"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6","Type":"ContainerDied","Data":"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"}
Nov 02 15:33:09.342515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.342501    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6" event={"ID":"ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6","Type":"ContainerDied","Data":"396a1cc41640f5fe99ebd1031b0a48838583dc37c5fb95d8e5a07eb56ce10947"}
Nov 02 15:33:09.355404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.352928    4470 scope.go:122] "RemoveContainer" containerID="c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"
Nov 02 15:33:09.355404 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:09.353176    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a\": container with ID starting with c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a not found: ID does not exist" containerID="c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"
Nov 02 15:33:09.355404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.353249    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a"} err="failed to get container status \"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a\": rpc error: code = NotFound desc = could not find container \"c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a\": container with ID starting with c497df2bc31e435463c81fad3ea36d90b96e6a98d327edb3ee2d5872bc52a28a not found: ID does not exist"
Nov 02 15:33:09.355404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.353270    4470 scope.go:122] "RemoveContainer" containerID="6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"
Nov 02 15:33:09.362821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.362802    4470 scope.go:122] "RemoveContainer" containerID="6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"
Nov 02 15:33:09.363133 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:09.363110    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16\": container with ID starting with 6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16 not found: ID does not exist" containerID="6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"
Nov 02 15:33:09.363245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.363137    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16"} err="failed to get container status \"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16\": rpc error: code = NotFound desc = could not find container \"6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16\": container with ID starting with 6a15f7e404a4848f31806ba81ec17ef5c22cb78e0df785448c9a60acf5957d16 not found: ID does not exist"
Nov 02 15:33:09.508080 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.508040    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:33:09.513943 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.513914    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-4c5b4e-5dddd9797f-xg7s6"]
Nov 02 15:33:09.636293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.634953    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:33:09.639628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.639588    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-4c5b4e-6579cc7f9f-wvstt"]
Nov 02 15:33:09.706199 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.706150    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1d28652-3d16-49be-8051-4a29139033e1" path="/var/lib/kubelet/pods/c1d28652-3d16-49be-8051-4a29139033e1/volumes"
Nov 02 15:33:09.706730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:09.706708    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" path="/var/lib/kubelet/pods/ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6/volumes"
Nov 02 15:33:13.309311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309273    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:33:13.309847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309833    4470 cpu_manager.go:414] "containerMap: removing container" podUID="aab15205-83f1-4672-93c8-61fb8a184d33" containerName="maas-api"
Nov 02 15:33:13.309847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309847    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aab15205-83f1-4672-93c8-61fb8a184d33" containerName="maas-api"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309862    4470 cpu_manager.go:414] "containerMap: removing container" podUID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" containerName="payload-pre-processing"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309867    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" containerName="payload-pre-processing"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309883    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309891    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309904    4470 cpu_manager.go:414] "containerMap: removing container" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309911    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309924    4470 cpu_manager.go:414] "containerMap: removing container" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309930    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309936    4470 cpu_manager.go:414] "containerMap: removing container" podUID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" containerName="maas-api"
Nov 02 15:33:13.309940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309940    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309948    4470 cpu_manager.go:414] "containerMap: removing container" podUID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309952    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309960    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309964    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309971    4470 cpu_manager.go:414] "containerMap: removing container" podUID="28c27b68-8549-42d5-841d-3c396bcaa889" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309975    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28c27b68-8549-42d5-841d-3c396bcaa889" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309981    4470 cpu_manager.go:414] "containerMap: removing container" podUID="79e443ad-b25f-4f75-9d0e-0ac310e712da" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309985    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79e443ad-b25f-4f75-9d0e-0ac310e712da" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309994    4470 cpu_manager.go:414] "containerMap: removing container" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.309998    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310003    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310008    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310019    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c1d28652-3d16-49be-8051-4a29139033e1" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310025    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1d28652-3d16-49be-8051-4a29139033e1" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310033    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310038    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310047    4470 cpu_manager.go:414] "containerMap: removing container" podUID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310053    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310058    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310062    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310071    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9979a001-a253-40e5-a602-8dd87cbd40b5" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310075    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9979a001-a253-40e5-a602-8dd87cbd40b5" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310082    4470 cpu_manager.go:414] "containerMap: removing container" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310086    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310092    4470 cpu_manager.go:414] "containerMap: removing container" podUID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310096    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310102    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310106    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310113    4470 cpu_manager.go:414] "containerMap: removing container" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310117    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310126    4470 cpu_manager.go:414] "containerMap: removing container" podUID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310130    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310137    4470 cpu_manager.go:414] "containerMap: removing container" podUID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310141    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310149    4470 cpu_manager.go:414] "containerMap: removing container" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310153    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310161    4470 cpu_manager.go:414] "containerMap: removing container" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310165    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310173    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310177    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310182    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310199    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310205    4470 cpu_manager.go:414] "containerMap: removing container" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310210    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310224    4470 cpu_manager.go:414] "containerMap: removing container" podUID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310229    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" containerName="payload-pre-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310237    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d78b2172-9204-4c57-89e4-b390cd1a14ef" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310242    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d78b2172-9204-4c57-89e4-b390cd1a14ef" containerName="payload-processing"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310249    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310253    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310258    4470 cpu_manager.go:414] "containerMap: removing container" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310263    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310271    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310276    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310282    4470 cpu_manager.go:414] "containerMap: removing container" podUID="970598d6-a53d-4128-9043-8af35fead769" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310303    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="970598d6-a53d-4128-9043-8af35fead769" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310310    4470 cpu_manager.go:414] "containerMap: removing container" podUID="2d116e99-553b-4ae3-a03b-d7582826b314" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310316    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d116e99-553b-4ae3-a03b-d7582826b314" containerName="maas-api"
Nov 02 15:33:13.310298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310326    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310331    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310339    4470 cpu_manager.go:414] "containerMap: removing container" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310344    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310429    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310437    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310442    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce021ff6-36f8-4e77-83f9-8c0c2ce99dd6" containerName="payload-pre-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310448    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310454    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310459    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b23d5ba-d054-4835-8f5a-e6886e76c1b3" containerName="payload-pre-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310466    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d78b2172-9204-4c57-89e4-b390cd1a14ef" containerName="payload-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310471    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce766b1b-5caa-44b2-8209-55baa7cb41e6" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310476    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310481    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="970598d6-a53d-4128-9043-8af35fead769" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310486    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d116e99-553b-4ae3-a03b-d7582826b314" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310492    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ddc1efd-e33d-44cc-862e-b2b8abee1c20" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310497    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="629aba4f-7644-4aec-88d3-f4559b6d3179" containerName="main"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310501    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="aab15205-83f1-4672-93c8-61fb8a184d33" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310507    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="af4c4fe7-8695-4c60-aac9-b7acd1ef5432" containerName="payload-pre-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310514    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310519    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a0cde38-b591-4c62-b753-1ccf1d17dd90" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310523    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b22ca92-967c-4aef-94af-7ce2f7fd017d" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310529    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310534    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="28c27b68-8549-42d5-841d-3c396bcaa889" containerName="payload-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310540    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="79e443ad-b25f-4f75-9d0e-0ac310e712da" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310546    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310551    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4b4f1f0-baa2-4f05-84af-f9ec411464a2" containerName="payload-pre-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310558    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1d28652-3d16-49be-8051-4a29139033e1" containerName="payload-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310564    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9400a595-d018-4feb-a913-9d2ec496cbf7" containerName="payload-pre-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310569    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="66757923-f4f5-4cde-9dc8-a9b0f3f1dc11" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310574    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a378458-a569-41a5-bcc0-98322d53a727" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310580    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9979a001-a253-40e5-a602-8dd87cbd40b5" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310585    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a33bb9e-1ddf-463a-ac22-d819b9c3181c" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310591    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ac4b109-4242-4ef3-bbe1-ca757eb493d5" containerName="maas-api"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310596    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae98f0a5-0b8b-454c-8f16-d4a0f949277a" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310602    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="6916f781-7088-43a3-811c-ab761ae73391" containerName="istio-proxy"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310609    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dcc35a0-0e4c-4457-8c2a-aaca2a928cf5" containerName="payload-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310614    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="248ccda4-84a5-4a23-9fb2-2f782e0e34c6" containerName="payload-processing"
Nov 02 15:33:13.312112 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.310620    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="bca3b944-22d2-4b34-921f-eccfb3b1090c" containerName="istio-proxy"
Nov 02 15:33:13.315407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.315386    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.367886 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.367661    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4e4683-openshift-default-dockercfg-nmpx9\""
Nov 02 15:33:13.420585 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.420554    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:33:13.430056 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430025    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430112    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430220    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430267    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430376    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgpn\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430401    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430533    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430603    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.430685 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.430660    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.538154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.538100    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.538429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.538223    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.539563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.539532    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.550213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.545584    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557334    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557425    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557465    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557561    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgpn\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557592    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557752    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.558842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.557833    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.560654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.560583    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.563471 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.561868    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.563471 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.562150    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.571831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.571797    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.573037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.573005    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.623809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.623771    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgpn\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.625388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.625358    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\") pod \"e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") " pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:13.629414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:13.629390    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:14.113458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.113432    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:33:14.115000 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:33:14.114973    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745c00bf_701e_461a_8558_28f1eb90e28c.slice/crio-288013d8bae25d3493f528b60f7296032f5ff9e0e7cd939866ea3a971810822d WatchSource:0}: Error finding container 288013d8bae25d3493f528b60f7296032f5ff9e0e7cd939866ea3a971810822d: Status 404 returned error can't find the container with id 288013d8bae25d3493f528b60f7296032f5ff9e0e7cd939866ea3a971810822d
Nov 02 15:33:14.117564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.117532    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:33:14.117649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.117595    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:33:14.117649 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.117624    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:33:14.408214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.408088    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerStarted","Data":"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"}
Nov 02 15:33:14.408214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.408154    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerStarted","Data":"288013d8bae25d3493f528b60f7296032f5ff9e0e7cd939866ea3a971810822d"}
Nov 02 15:33:14.531116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.531057    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podStartSLOduration=1.531040672 podStartE2EDuration="1.531040672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:33:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:33:14.528302733 +0000 UTC m=+2200.966204874" watchObservedRunningTime="2026-11-02 15:33:14.531040672 +0000 UTC m=+2200.968942812"
Nov 02 15:33:14.629479 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:14.629441    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:15.655155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:15.653412    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:16.421272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:16.421179    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:16.422564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:16.422538    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:33:20.536262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.536219    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:33:20.537028 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.537005    4470 cpu_manager.go:414] "containerMap: removing container" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:20.537129 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.537029    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12f8cb24-efd6-4fbc-b353-86bb64d2c71a" containerName="istio-proxy"
Nov 02 15:33:20.537255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.537157    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3ab13b7-1bf8-4a67-86fc-bdb411713174" containerName="istio-proxy"
Nov 02 15:33:20.537255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.537171    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c780baf4-d4aa-4f00-b000-589f3b6a197f" containerName="istio-proxy"
Nov 02 15:33:20.537255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.537183    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="93c7016c-1dc1-4269-9a32-ed3c1c264e1a" containerName="istio-proxy"
Nov 02 15:33:20.540970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.540949    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.598521 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.598487    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-de9c4ba4-serving-cert\""
Nov 02 15:33:20.646410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.646374    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:33:20.646959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.646935    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvxhw\" (UniqueName: \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.647010 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.646983    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.647163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.647141    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.647306 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.647281    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.749068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.749033    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.749286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.749142    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvxhw\" (UniqueName: \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.749286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.749169    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.749286 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:20.749219    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-serving-cert: secret "maas-api-e2e-mt-de9c4ba4-serving-cert" not found
Nov 02 15:33:20.749430 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:20.749303    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls podName:f814477a-361f-45a2-ba22-c204953004c9 nodeName:}" failed. No retries permitted until 2026-11-02 15:33:21.249283269 +0000 UTC m=+2207.687185403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls") pod "maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" (UID: "f814477a-361f-45a2-ba22-c204953004c9") : secret "maas-api-e2e-mt-de9c4ba4-serving-cert" not found
Nov 02 15:33:20.749430 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.749327    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.749958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.749937    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.751891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.751873    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:20.810982 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:20.810890    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvxhw\" (UniqueName: \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:21.265004 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:21.264960    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:21.267703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:21.267675    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") pod \"maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:21.456047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:21.456011    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:21.838642 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:21.838539    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:33:21.843243 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:33:21.843205    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf814477a_361f_45a2_ba22_c204953004c9.slice/crio-45d9e9792d5f825b078b34633a88bf4e0ae5f4c8cef78d61d9cec3a9a78a5615 WatchSource:0}: Error finding container 45d9e9792d5f825b078b34633a88bf4e0ae5f4c8cef78d61d9cec3a9a78a5615: Status 404 returned error can't find the container with id 45d9e9792d5f825b078b34633a88bf4e0ae5f4c8cef78d61d9cec3a9a78a5615
Nov 02 15:33:21.844130 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:21.844111    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:33:22.463465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:22.463419    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" event={"ID":"f814477a-361f-45a2-ba22-c204953004c9","Type":"ContainerStarted","Data":"45d9e9792d5f825b078b34633a88bf4e0ae5f4c8cef78d61d9cec3a9a78a5615"}
Nov 02 15:33:23.471242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:23.471199    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" event={"ID":"f814477a-361f-45a2-ba22-c204953004c9","Type":"ContainerStarted","Data":"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"}
Nov 02 15:33:23.471766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:23.471404    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:23.589375 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:23.589312    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" podStartSLOduration=2.706228299 podStartE2EDuration="3.589293575s" totalImagesPullingTime="883.065276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:33:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:33:23.588986724 +0000 UTC m=+2210.026888866" watchObservedRunningTime="2026-11-02 15:33:23.589293575 +0000 UTC m=+2210.027195717"
Nov 02 15:33:29.483953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:29.483918    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:33:33.573783 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:33.573735    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:33:44.704323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:44.704286    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:33:44.704838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:44.704492    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" podUID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" containerName="maas-api" containerID="cri-o://655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d" gracePeriod=30
Nov 02 15:33:45.026416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.026385    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:33:45.103310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.097584    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\" (UniqueName: \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\") pod \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") "
Nov 02 15:33:45.103310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.097726    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") pod \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") "
Nov 02 15:33:45.103310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.097829    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\") pod \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") "
Nov 02 15:33:45.103310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.097924    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\") pod \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\" (UID: \"416fb9f2-d1f2-443d-a079-90df3a9a52e3\") "
Nov 02 15:33:45.110451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.110078    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca" pod "416fb9f2-d1f2-443d-a079-90df3a9a52e3" (UID: "416fb9f2-d1f2-443d-a079-90df3a9a52e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:33:45.120603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.120362    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls" pod "416fb9f2-d1f2-443d-a079-90df3a9a52e3" (UID: "416fb9f2-d1f2-443d-a079-90df3a9a52e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:33:45.120603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.120495    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52" pod "416fb9f2-d1f2-443d-a079-90df3a9a52e3" (UID: "416fb9f2-d1f2-443d-a079-90df3a9a52e3"). InnerVolumeSpecName "kube-api-access-bms52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:33:45.120603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.120573    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs" pod "416fb9f2-d1f2-443d-a079-90df3a9a52e3" (UID: "416fb9f2-d1f2-443d-a079-90df3a9a52e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:33:45.200677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.200633    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:45.200677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.200663    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/416fb9f2-d1f2-443d-a079-90df3a9a52e3-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:45.200677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.200673    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/416fb9f2-d1f2-443d-a079-90df3a9a52e3-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:45.200677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.200682    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bms52\" (UniqueName: \"kubernetes.io/projected/416fb9f2-d1f2-443d-a079-90df3a9a52e3-kube-api-access-bms52\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:45.617628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.617591    4470 generic.go:356] "Generic (PLEG): container finished" podID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" containerID="655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d" exitCode=0
Nov 02 15:33:45.617846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.617653    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"
Nov 02 15:33:45.617846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.617672    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" event={"ID":"416fb9f2-d1f2-443d-a079-90df3a9a52e3","Type":"ContainerDied","Data":"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"}
Nov 02 15:33:45.617846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.617731    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs" event={"ID":"416fb9f2-d1f2-443d-a079-90df3a9a52e3","Type":"ContainerDied","Data":"a7c0c7586cc1a1d100720fb9a5872db990d73cd8715578a4a31a781f06531262"}
Nov 02 15:33:45.617846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.617754    4470 scope.go:122] "RemoveContainer" containerID="655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"
Nov 02 15:33:45.631867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.631844    4470 scope.go:122] "RemoveContainer" containerID="655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"
Nov 02 15:33:45.632438 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:45.632410    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d\": container with ID starting with 655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d not found: ID does not exist" containerID="655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"
Nov 02 15:33:45.632544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.632448    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d"} err="failed to get container status \"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d\": rpc error: code = NotFound desc = could not find container \"655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d\": container with ID starting with 655b150d9eca1b7a01ecdd98b6d76e72cb71fd2a3ffc51975d437d5ce267a75d not found: ID does not exist"
Nov 02 15:33:45.784254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.784217    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:33:45.788941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:45.788915    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4c5b4e-85cfbbcb55-pwrvs"]
Nov 02 15:33:47.707035 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:47.706993    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" path="/var/lib/kubelet/pods/416fb9f2-d1f2-443d-a079-90df3a9a52e3/volumes"
Nov 02 15:33:54.104363 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:54.104323    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:33:54.104857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:54.104555    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy" containerID="cri-o://53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024" gracePeriod=30
Nov 02 15:33:59.374438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.374365    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:33:59.399705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399663    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.399883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399775    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.399883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399813    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399885    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399913    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.399939    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400047    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400077    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:33:59.400325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400106    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400152    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\") pod \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\" (UID: \"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23\") "
Nov 02 15:33:59.400325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400232    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:33:59.400542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400321    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:33:59.400542 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400472    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:33:59.400684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400661    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:33:59.400741 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400723    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.400784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400740    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.400784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400750    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.400784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.400767    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.402487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.402464    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:33:59.402564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.402500    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:33:59.402564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.402527    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "kube-api-access-mb7vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:33:59.402564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.402527    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo" pod "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" (UID: "eb8938a9-ac6f-4754-95bf-8ffc8dec5d23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:33:59.502426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.502390    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.502426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.502424    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb7vr\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-kube-api-access-mb7vr\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.502426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.502435    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.502687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.502444    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.502687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.502453    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:33:59.598564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.598526    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:33:59.714162 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.714130    4470 generic.go:356] "Generic (PLEG): container finished" podID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerID="53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024" exitCode=0
Nov 02 15:33:59.714360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.714172    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerDied","Data":"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"}
Nov 02 15:33:59.714360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.714236    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"
Nov 02 15:33:59.714360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.714250    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8" event={"ID":"eb8938a9-ac6f-4754-95bf-8ffc8dec5d23","Type":"ContainerDied","Data":"9a8b4a57e4f1e085c9bcf3235a3eb910509cf0b79b07fbc9c5c88de8d94551d9"}
Nov 02 15:33:59.714360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.714281    4470 scope.go:122] "RemoveContainer" containerID="53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"
Nov 02 15:33:59.725864 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.725843    4470 scope.go:122] "RemoveContainer" containerID="88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"
Nov 02 15:33:59.735320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.735299    4470 scope.go:122] "RemoveContainer" containerID="53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"
Nov 02 15:33:59.735648 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:59.735628    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024\": container with ID starting with 53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024 not found: ID does not exist" containerID="53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"
Nov 02 15:33:59.735742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.735662    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024"} err="failed to get container status \"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024\": rpc error: code = NotFound desc = could not find container \"53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024\": container with ID starting with 53f32ddd3f6d7a7f9d9c7dd8e8a480cb8f8d4704482b573cb4d04acd1d0a4024 not found: ID does not exist"
Nov 02 15:33:59.735742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.735693    4470 scope.go:122] "RemoveContainer" containerID="88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"
Nov 02 15:33:59.735973 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:33:59.735950    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5\": container with ID starting with 88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5 not found: ID does not exist" containerID="88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"
Nov 02 15:33:59.736021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.735984    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5"} err="failed to get container status \"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5\": rpc error: code = NotFound desc = could not find container \"88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5\": container with ID starting with 88875627ee240a7136f787366bd9be813f9de03798308650912b2bba5fac35f5 not found: ID does not exist"
Nov 02 15:33:59.880993 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.880954    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:33:59.885754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:33:59.885726    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-4c5b4e-openshift-default-7d9554dfb5-k4bg8"]
Nov 02 15:34:00.526881 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.526848    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"]
Nov 02 15:34:00.527417 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527402    4470 cpu_manager.go:414] "containerMap: removing container" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:00.527467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527418    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:00.527467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527453    4470 cpu_manager.go:414] "containerMap: removing container" podUID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" containerName="maas-api"
Nov 02 15:34:00.527467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527459    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" containerName="maas-api"
Nov 02 15:34:00.527575 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527539    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:00.527575 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.527551    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="416fb9f2-d1f2-443d-a079-90df3a9a52e3" containerName="maas-api"
Nov 02 15:34:00.530875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.530856    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616134    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e86ef4a-26ce-435d-a63f-f951e10ee840-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616287    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616316    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616364    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbk5b\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-kube-api-access-xbk5b\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616476    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616502    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616526    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616673    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.616746 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.616693    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717444    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717484    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717508    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e86ef4a-26ce-435d-a63f-f951e10ee840-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717773    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717892 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717843    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717908    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-credential-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.717947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.717937    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbk5b\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-kube-api-access-xbk5b\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718015    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718042    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718087    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-data\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718149    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-socket\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718251    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718299    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e86ef4a-26ce-435d-a63f-f951e10ee840-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.718875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.718849    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-workload-certs\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.720339 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.720316    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-envoy\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.721114 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.721089    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.734839 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.734815    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"]
Nov 02 15:34:00.779809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.779728    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-istio-token\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.780005 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.779982    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbk5b\" (UniqueName: \"kubernetes.io/projected/9e86ef4a-26ce-435d-a63f-f951e10ee840-kube-api-access-xbk5b\") pod \"maas-default-gateway-openshift-default-75fd9f5d47-5j55g\" (UID: \"9e86ef4a-26ce-435d-a63f-f951e10ee840\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:00.845326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:00.845285    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:01.281093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.281058    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"]
Nov 02 15:34:01.318466 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:34:01.318432    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e86ef4a_26ce_435d_a63f_f951e10ee840.slice/crio-0edf378346be34a7be948d94e20124cfc2603e312325680d3f89f9f198750d92 WatchSource:0}: Error finding container 0edf378346be34a7be948d94e20124cfc2603e312325680d3f89f9f198750d92: Status 404 returned error can't find the container with id 0edf378346be34a7be948d94e20124cfc2603e312325680d3f89f9f198750d92
Nov 02 15:34:01.320981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.320937    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:01.321092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.321033    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:01.321092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.321084    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:01.707593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.707558    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" path="/var/lib/kubelet/pods/eb8938a9-ac6f-4754-95bf-8ffc8dec5d23/volumes"
Nov 02 15:34:01.730660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.730610    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" event={"ID":"9e86ef4a-26ce-435d-a63f-f951e10ee840","Type":"ContainerStarted","Data":"d6e6bd9dc243669504d9811d4ac254e17714bb221a3f0b0e7d4c40e6330c09d4"}
Nov 02 15:34:01.730830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.730693    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" event={"ID":"9e86ef4a-26ce-435d-a63f-f951e10ee840","Type":"ContainerStarted","Data":"0edf378346be34a7be948d94e20124cfc2603e312325680d3f89f9f198750d92"}
Nov 02 15:34:01.845688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.845649    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:01.852486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:01.852428    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podStartSLOduration=1.852414934 podStartE2EDuration="1.852414934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:34:01.851412409 +0000 UTC m=+2248.289314556" watchObservedRunningTime="2026-11-02 15:34:01.852414934 +0000 UTC m=+2248.290317075"
Nov 02 15:34:02.847787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:02.847754    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:02.848244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:02.847806    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:03.566843 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:34:03.566795    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:34:03.846539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:03.846438    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:03.846539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:03.846498    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:04.422067 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.422029    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:34:04.422854 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.422831    4470 cpu_manager.go:414] "containerMap: removing container" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:04.422961 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.422854    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:04.423061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.423046    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb8938a9-ac6f-4754-95bf-8ffc8dec5d23" containerName="istio-proxy"
Nov 02 15:34:04.426713 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.426692    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465623    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465672    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465695    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465865    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465897    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.465924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465920    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.466081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.465954    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h456l\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.466081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.466049    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.466203 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.466171    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.489226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.488566    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-de9c4ba4-openshift-default-dockercfg-7b79t\""
Nov 02 15:34:04.542223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.542172    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:34:04.567755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.567712    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.567976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.567849    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.567976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.567897    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.567976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.567926    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568011    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568040    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568386    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568418 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568401    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568455    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h456l\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568490    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.568628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568544    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.569008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.568983    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.569103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.569075    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.569180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.569096    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.596701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.596659    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.597795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.597767    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.629350 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.629313    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.629515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.629425    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h456l\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.741407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.741364    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:04.845816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.845774    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:04.846008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:04.845838    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:05.203331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.203301    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:34:05.205723 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:34:05.205696    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a06ba5_14b6_476c_9cee_2d45afa33684.slice/crio-95fd9003a16e0bb54f92462ee22d71e16a85ea3e0e2bdd7d76c4d676020decf4 WatchSource:0}: Error finding container 95fd9003a16e0bb54f92462ee22d71e16a85ea3e0e2bdd7d76c4d676020decf4: Status 404 returned error can't find the container with id 95fd9003a16e0bb54f92462ee22d71e16a85ea3e0e2bdd7d76c4d676020decf4
Nov 02 15:34:05.208280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.208245    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:05.208342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.208313    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:05.208379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.208344    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:05.758771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.758728    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" event={"ID":"d6a06ba5-14b6-476c-9cee-2d45afa33684","Type":"ContainerStarted","Data":"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"}
Nov 02 15:34:05.759267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.758799    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" event={"ID":"d6a06ba5-14b6-476c-9cee-2d45afa33684","Type":"ContainerStarted","Data":"95fd9003a16e0bb54f92462ee22d71e16a85ea3e0e2bdd7d76c4d676020decf4"}
Nov 02 15:34:05.846384 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.846343    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:05.846574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.846411    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:05.884670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:05.884603    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podStartSLOduration=1.884582223 podStartE2EDuration="1.884582223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:34:05.879728262 +0000 UTC m=+2252.317630406" watchObservedRunningTime="2026-11-02 15:34:05.884582223 +0000 UTC m=+2252.322484365"
Nov 02 15:34:06.743488 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:06.743416    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:06.745284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:06.745253    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:06.745438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:06.745312    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:06.846025 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:06.845987    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:06.846518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:06.846047    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:07.108267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.106638    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:34:07.168154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.165828    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.241214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.241141    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:34:07.244275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.243959    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-4e4683-serving-cert\""
Nov 02 15:34:07.317214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.317002    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q4xc\" (UniqueName: \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.317214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.317138    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.317482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.317231    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.317482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.317403    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.434089    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.434146    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q4xc\" (UniqueName: \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.434277    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:34:07.434324    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-serving-cert: secret "maas-api-e2e-inf-b-4e4683-serving-cert" not found
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.434350    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.435136 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:34:07.434405    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls podName:32e6d9b0-9e98-4839-a167-6486d5f27dae nodeName:}" failed. No retries permitted until 2026-11-02 15:34:07.934383427 +0000 UTC m=+2254.372285561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls") pod "maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" (UID: "32e6d9b0-9e98-4839-a167-6486d5f27dae") : secret "maas-api-e2e-inf-b-4e4683-serving-cert" not found
Nov 02 15:34:07.459234 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.457534    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.466347 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.464307    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.518214 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.517819    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q4xc\" (UniqueName: \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.744292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.744083    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:07.744292 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.744148    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:07.846093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.845683    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:07.846093 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.845747    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:07.946579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.946526    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:07.951776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:07.951746    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") pod \"maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:08.157269 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.155887    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:08.742250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.742179    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:08.742452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.742281    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:08.796605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.796468    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:34:08.805136 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:34:08.805101    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e6d9b0_9e98_4839_a167_6486d5f27dae.slice/crio-f3fe7893995de3216a40417bf37a1cc7a9118ea6691c48781912d20d8cfa0dff WatchSource:0}: Error finding container f3fe7893995de3216a40417bf37a1cc7a9118ea6691c48781912d20d8cfa0dff: Status 404 returned error can't find the container with id f3fe7893995de3216a40417bf37a1cc7a9118ea6691c48781912d20d8cfa0dff
Nov 02 15:34:08.846115 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.846072    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:08.846553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:08.846143    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:09.741884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:09.741852    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:09.742011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:09.741928    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:09.815045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:09.815006    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" event={"ID":"32e6d9b0-9e98-4839-a167-6486d5f27dae","Type":"ContainerStarted","Data":"f3fe7893995de3216a40417bf37a1cc7a9118ea6691c48781912d20d8cfa0dff"}
Nov 02 15:34:09.846737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:09.846703    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused" start-of-body=
Nov 02 15:34:09.847149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:09.846780    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": dial tcp 10.133.0.138:15021: connect: connection refused"
Nov 02 15:34:10.742352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:10.741746    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:10.742352 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:10.741813    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:10.822948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:10.822901    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" event={"ID":"32e6d9b0-9e98-4839-a167-6486d5f27dae","Type":"ContainerStarted","Data":"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"}
Nov 02 15:34:10.823171 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:10.823065    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:11.041215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.041092    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:11.041655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.041530    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:11.042524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.042506    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g"
Nov 02 15:34:11.197677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.197616    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" podStartSLOduration=3.266586044 podStartE2EDuration="4.197600879s" totalImagesPullingTime="931.014835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:34:10.96939575 +0000 UTC m=+2257.407297903" watchObservedRunningTime="2026-11-02 15:34:11.197600879 +0000 UTC m=+2257.635503069"
Nov 02 15:34:11.742684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.742636    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused" start-of-body=
Nov 02 15:34:11.742900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:11.742713    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.139:15021/healthz/ready\": dial tcp 10.133.0.139:15021: connect: connection refused"
Nov 02 15:34:12.747053 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:12.747015    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:12.747571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:12.747453    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:12.748171 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:12.748151    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:34:14.619598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:14.619552    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" containerID="cri-o://7055eb4152c677c5dd782153dc76fa36dce9c02b2d1be7c62c6c28015fe5dc1f" gracePeriod=45
Nov 02 15:34:14.739769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:14.739733    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": dial tcp 10.133.0.58:8000: connect: connection refused"
Nov 02 15:34:14.854124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:14.854090    4470 generic.go:356] "Generic (PLEG): container finished" podID="34303c70-a1e0-496c-838c-60cc45cdccea" containerID="7055eb4152c677c5dd782153dc76fa36dce9c02b2d1be7c62c6c28015fe5dc1f" exitCode=0
Nov 02 15:34:14.854324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:14.854218    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" event={"ID":"34303c70-a1e0-496c-838c-60cc45cdccea","Type":"ContainerDied","Data":"7055eb4152c677c5dd782153dc76fa36dce9c02b2d1be7c62c6c28015fe5dc1f"}
Nov 02 15:34:14.908322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:14.908293    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:34:15.006837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.006800    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.006943    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\" (UniqueName: \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.006978    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\" (UniqueName: \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007090    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007128    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007226    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\") pod \"34303c70-a1e0-496c-838c-60cc45cdccea\" (UID: \"34303c70-a1e0-496c-838c-60cc45cdccea\") "
Nov 02 15:34:15.007920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007461    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:34:15.007920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007672    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:34:15.007920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007762    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:34:15.007920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007869    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.007920 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.007893    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.009598 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.009568    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:34:15.009761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.009739    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "kube-api-access-g7297". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:34:15.009875 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.009859    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm" pod "34303c70-a1e0-496c-838c-60cc45cdccea" (UID: "34303c70-a1e0-496c-838c-60cc45cdccea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:34:15.109538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.109496    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34303c70-a1e0-496c-838c-60cc45cdccea-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.109538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.109525    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7297\" (UniqueName: \"kubernetes.io/projected/34303c70-a1e0-496c-838c-60cc45cdccea-kube-api-access-g7297\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.109538 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.109538    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.109867 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.109550    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34303c70-a1e0-496c-838c-60cc45cdccea-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:34:15.862202 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.862163    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"
Nov 02 15:34:15.862695 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.862221    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz" event={"ID":"34303c70-a1e0-496c-838c-60cc45cdccea","Type":"ContainerDied","Data":"aec2a7df34e9bbf4df6ce75eab59ad4aa6ec0cfad12c90ffa056200a94e48b8a"}
Nov 02 15:34:15.862695 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:15.862273    4470 scope.go:122] "RemoveContainer" containerID="7055eb4152c677c5dd782153dc76fa36dce9c02b2d1be7c62c6c28015fe5dc1f"
Nov 02 15:34:16.027938 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:16.027850    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:34:16.077227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:16.077181    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-42555d/premium-w3-42555d-kserve-f8b5c7668-r6jhz"]
Nov 02 15:34:16.837730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:16.837699    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:34:17.707467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:17.707428    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" path="/var/lib/kubelet/pods/34303c70-a1e0-496c-838c-60cc45cdccea/volumes"
Nov 02 15:34:19.619999 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.619955    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:34:19.620630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.620614    4470 cpu_manager.go:414] "containerMap: removing container" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main"
Nov 02 15:34:19.620671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.620631    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main"
Nov 02 15:34:19.620784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.620773    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="34303c70-a1e0-496c-838c-60cc45cdccea" containerName="main"
Nov 02 15:34:19.625738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.625717    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.674733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.674696    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.674905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.674744    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.674905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.674873    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.674983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.674947    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.675021 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.675005    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.675103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.675087    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzf67\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.675151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.675136    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.675223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.675206    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.675267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.675240    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.776643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776603    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.776853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776673    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzf67\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.776853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776829    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.776989 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776899    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.776989 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776936    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.776990    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777038    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777140    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777223 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777166    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777264    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777507    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777662    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.777936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777902    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.778058 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.777958    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.780007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.779971    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.780105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.780022    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.859168 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.859130    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.859382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.859254    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzf67\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\") pod \"e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") " pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:19.862766 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.862736    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:34:19.939768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:19.939734    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:20.379928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.379904    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:34:20.382085 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:34:20.381872    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009fd0fc_b76d_4502_9cfe_0de41af187b6.slice/crio-4534cfb50d0f4361ae42eba46dd9532867536a9e8e34d2c58ffcd8d0c185e899 WatchSource:0}: Error finding container 4534cfb50d0f4361ae42eba46dd9532867536a9e8e34d2c58ffcd8d0c185e899: Status 404 returned error can't find the container with id 4534cfb50d0f4361ae42eba46dd9532867536a9e8e34d2c58ffcd8d0c185e899
Nov 02 15:34:20.385067 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.385036    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:20.385142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.385102    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:20.385142 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.385132    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:20.908498 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.908448    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" event={"ID":"009fd0fc-b76d-4502-9cfe-0de41af187b6","Type":"ContainerStarted","Data":"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"}
Nov 02 15:34:20.908498 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.908503    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" event={"ID":"009fd0fc-b76d-4502-9cfe-0de41af187b6","Type":"ContainerStarted","Data":"4534cfb50d0f4361ae42eba46dd9532867536a9e8e34d2c58ffcd8d0c185e899"}
Nov 02 15:34:20.940620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:20.940585    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:21.095213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:21.083523    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podStartSLOduration=2.083503067 podStartE2EDuration="2.083503067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:34:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:34:21.081706519 +0000 UTC m=+2267.519608672" watchObservedRunningTime="2026-11-02 15:34:21.083503067 +0000 UTC m=+2267.521405209"
Nov 02 15:34:21.943343 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:21.943295    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Nov 02 15:34:21.943730 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:21.943369    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Nov 02 15:34:22.940245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:22.940171    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Nov 02 15:34:22.940469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:22.940287    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Nov 02 15:34:23.941043 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:23.941006    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Nov 02 15:34:23.941461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:23.941076    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Nov 02 15:34:24.940725 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:24.940687    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Nov 02 15:34:24.940939 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:24.940742    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Nov 02 15:34:25.940565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:25.940529    4470 patch_prober.go:29] interesting pod/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused" start-of-body=
Nov 02 15:34:25.940950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:25.940584    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.141:15021/healthz/ready\": dial tcp 10.133.0.141:15021: connect: connection refused"
Nov 02 15:34:27.302905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:27.302870    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:27.303668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:27.303631    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:27.305178 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:27.305127    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:34:33.566105 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:34:33.566063    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:34:42.099936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.099846    4470 generic.go:356] "Generic (PLEG): container finished" podID="745c00bf-701e-461a-8558-28f1eb90e28c" containerID="4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e" exitCode=0
Nov 02 15:34:42.099936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.099915    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerDied","Data":"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"}
Nov 02 15:34:42.100435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.100404    4470 scope.go:122] "RemoveContainer" containerID="4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"
Nov 02 15:34:42.102067 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.102035    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:42.102177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.102112    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:42.102177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:42.102148    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:34:43.110947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:43.110903    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerStarted","Data":"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"}
Nov 02 15:34:43.629621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:43.629586    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:34:43.630624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:43.630597    4470 patch_prober.go:29] interesting pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Nov 02 15:34:43.630787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:43.630659    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Nov 02 15:34:44.629733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:44.629699    4470 patch_prober.go:29] interesting pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Nov 02 15:34:44.630147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:44.629751    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Nov 02 15:34:45.630689 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:45.630642    4470 patch_prober.go:29] interesting pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Nov 02 15:34:45.631103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:45.630730    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Nov 02 15:34:46.404293 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:46.404172    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:34:46.421634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:46.421598    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:34:46.454054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:46.454023    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:34:46.630092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:46.630044    4470 patch_prober.go:29] interesting pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Nov 02 15:34:46.630357 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:46.630128    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Nov 02 15:34:47.630628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:47.630589    4470 patch_prober.go:29] interesting pod/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused" start-of-body=
Nov 02 15:34:47.631061 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:47.630655    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.136:15021/healthz/ready\": dial tcp 10.133.0.136:15021: connect: connection refused"
Nov 02 15:34:48.635723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:48.635688    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:34:48.637181 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:34:48.637154    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:35:01.427098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.427051    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main" containerID="cri-o://6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f" gracePeriod=45
Nov 02 15:35:01.499605 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.499565    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main" containerID="cri-o://7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf" gracePeriod=45
Nov 02 15:35:01.804037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.804000    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:35:01.807331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.807304    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:35:01.939045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939011    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939063    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\" (UniqueName: \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.939271 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939176    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939305    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.939408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939335    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.939408 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939386    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.939572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939454    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\" (UniqueName: \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939494    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\" (UniqueName: \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939526    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.939572 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939562    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939625    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.939798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939634    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.939798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939729    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\") pod \"679279a9-76bb-49aa-a850-04469ea87fcf\" (UID: \"679279a9-76bb-49aa-a850-04469ea87fcf\") "
Nov 02 15:35:01.939798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939788    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\") pod \"9a915c87-81cc-4937-b9fc-c7d78ff67832\" (UID: \"9a915c87-81cc-4937-b9fc-c7d78ff67832\") "
Nov 02 15:35:01.940011 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.939831    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.940405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940297    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.940405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940302    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.940590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940508    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.940590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940537    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:01.940590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940559    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:01.940590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940574    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:01.940590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940590    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:01.940769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.940605    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:01.942087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942056    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "kube-api-access-hqnjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:01.942310 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942284    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:01.942414 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942378    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm" pod "9a915c87-81cc-4937-b9fc-c7d78ff67832" (UID: "9a915c87-81cc-4937-b9fc-c7d78ff67832"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.942530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942510    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:01.942712 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942691    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "kube-api-access-s9xwm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:01.942776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:01.942762    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs" pod "679279a9-76bb-49aa-a850-04469ea87fcf" (UID: "679279a9-76bb-49aa-a850-04469ea87fcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:02.041484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041449    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a915c87-81cc-4937-b9fc-c7d78ff67832-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041481    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/679279a9-76bb-49aa-a850-04469ea87fcf-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041491    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9xwm\" (UniqueName: \"kubernetes.io/projected/679279a9-76bb-49aa-a850-04469ea87fcf-kube-api-access-s9xwm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041501    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/679279a9-76bb-49aa-a850-04469ea87fcf-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041509    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041517    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a915c87-81cc-4937-b9fc-c7d78ff67832-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.041745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.041525    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqnjj\" (UniqueName: \"kubernetes.io/projected/9a915c87-81cc-4937-b9fc-c7d78ff67832-kube-api-access-hqnjj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:02.371814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.371714    4470 generic.go:356] "Generic (PLEG): container finished" podID="679279a9-76bb-49aa-a850-04469ea87fcf" containerID="6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f" exitCode=0
Nov 02 15:35:02.371814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.371765    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"
Nov 02 15:35:02.371814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.371798    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" event={"ID":"679279a9-76bb-49aa-a850-04469ea87fcf","Type":"ContainerDied","Data":"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"}
Nov 02 15:35:02.372108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.371843    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch" event={"ID":"679279a9-76bb-49aa-a850-04469ea87fcf","Type":"ContainerDied","Data":"df517c7ea474834bd35cfec13b2fe7d03703818206e3e68d861cf081bf11333c"}
Nov 02 15:35:02.372108 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.371862    4470 scope.go:122] "RemoveContainer" containerID="6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"
Nov 02 15:35:02.373455 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.373428    4470 generic.go:356] "Generic (PLEG): container finished" podID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerID="7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf" exitCode=0
Nov 02 15:35:02.373587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.373472    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"
Nov 02 15:35:02.373587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.373508    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" event={"ID":"9a915c87-81cc-4937-b9fc-c7d78ff67832","Type":"ContainerDied","Data":"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"}
Nov 02 15:35:02.373587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.373568    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj" event={"ID":"9a915c87-81cc-4937-b9fc-c7d78ff67832","Type":"ContainerDied","Data":"13c01a51daf7776cf2831429be6590752caf87073c28813bb1d93adaa47e2d26"}
Nov 02 15:35:02.383328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.383304    4470 scope.go:122] "RemoveContainer" containerID="6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"
Nov 02 15:35:02.383689 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:02.383668    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f\": container with ID starting with 6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f not found: ID does not exist" containerID="6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"
Nov 02 15:35:02.383754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.383700    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f"} err="failed to get container status \"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f\": rpc error: code = NotFound desc = could not find container \"6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f\": container with ID starting with 6f592e12c2901aaa9e31fb0d819409c14e370d377d9c7e90cfab7bf42071405f not found: ID does not exist"
Nov 02 15:35:02.383754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.383720    4470 scope.go:122] "RemoveContainer" containerID="7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"
Nov 02 15:35:02.393126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.393101    4470 scope.go:122] "RemoveContainer" containerID="7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"
Nov 02 15:35:02.393504 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:02.393482    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf\": container with ID starting with 7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf not found: ID does not exist" containerID="7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"
Nov 02 15:35:02.393587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.393519    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf"} err="failed to get container status \"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf\": rpc error: code = NotFound desc = could not find container \"7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf\": container with ID starting with 7ed9983dae41687775467ff7e2d967746490815372eeaf03ca48029bad640baf not found: ID does not exist"
Nov 02 15:35:02.539060 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.539022    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:35:02.545896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.545863    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-w2-de35cd-kserve-c459c89b7-jwbtj"]
Nov 02 15:35:02.661397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.661304    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:35:02.667609 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:02.667578    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-de35cd/distinct-2-w2-de35cd-kserve-68fdc7cf5b-fqfch"]
Nov 02 15:35:03.566648 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:03.566608    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:35:03.707342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:03.707305    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" path="/var/lib/kubelet/pods/679279a9-76bb-49aa-a850-04469ea87fcf/volumes"
Nov 02 15:35:03.707872 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:03.707856    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" path="/var/lib/kubelet/pods/9a915c87-81cc-4937-b9fc-c7d78ff67832/volumes"
Nov 02 15:35:04.481147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.481052    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:35:04.481655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.481626    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" podUID="f814477a-361f-45a2-ba22-c204953004c9" containerName="maas-api" containerID="cri-o://d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289" gracePeriod=30
Nov 02 15:35:04.827758 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.827728    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:35:04.989154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.989109    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") pod \"f814477a-361f-45a2-ba22-c204953004c9\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") "
Nov 02 15:35:04.989397 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.989353    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\" (UniqueName: \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\") pod \"f814477a-361f-45a2-ba22-c204953004c9\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") "
Nov 02 15:35:04.989480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.989453    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\") pod \"f814477a-361f-45a2-ba22-c204953004c9\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") "
Nov 02 15:35:04.989545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.989524    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\") pod \"f814477a-361f-45a2-ba22-c204953004c9\" (UID: \"f814477a-361f-45a2-ba22-c204953004c9\") "
Nov 02 15:35:04.990736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.990691    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca" pod "f814477a-361f-45a2-ba22-c204953004c9" (UID: "f814477a-361f-45a2-ba22-c204953004c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:35:04.992570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:04.992542    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw" pod "f814477a-361f-45a2-ba22-c204953004c9" (UID: "f814477a-361f-45a2-ba22-c204953004c9"). InnerVolumeSpecName "kube-api-access-hvxhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:05.007795 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.007754    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs" pod "f814477a-361f-45a2-ba22-c204953004c9" (UID: "f814477a-361f-45a2-ba22-c204953004c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:05.008105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.008079    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls" pod "f814477a-361f-45a2-ba22-c204953004c9" (UID: "f814477a-361f-45a2-ba22-c204953004c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:05.092616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.092519    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f814477a-361f-45a2-ba22-c204953004c9-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:05.092616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.092568    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:05.092616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.092586    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f814477a-361f-45a2-ba22-c204953004c9-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:05.092616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.092602    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvxhw\" (UniqueName: \"kubernetes.io/projected/f814477a-361f-45a2-ba22-c204953004c9-kube-api-access-hvxhw\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:05.406549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.406457    4470 generic.go:356] "Generic (PLEG): container finished" podID="f814477a-361f-45a2-ba22-c204953004c9" containerID="d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289" exitCode=0
Nov 02 15:35:05.406549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.406522    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"
Nov 02 15:35:05.406549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.406533    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" event={"ID":"f814477a-361f-45a2-ba22-c204953004c9","Type":"ContainerDied","Data":"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"}
Nov 02 15:35:05.406876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.406591    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw" event={"ID":"f814477a-361f-45a2-ba22-c204953004c9","Type":"ContainerDied","Data":"45d9e9792d5f825b078b34633a88bf4e0ae5f4c8cef78d61d9cec3a9a78a5615"}
Nov 02 15:35:05.406876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.406616    4470 scope.go:122] "RemoveContainer" containerID="d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"
Nov 02 15:35:05.417380 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.417362    4470 scope.go:122] "RemoveContainer" containerID="d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"
Nov 02 15:35:05.417670 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:05.417652    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289\": container with ID starting with d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289 not found: ID does not exist" containerID="d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"
Nov 02 15:35:05.417722 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.417679    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289"} err="failed to get container status \"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289\": rpc error: code = NotFound desc = could not find container \"d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289\": container with ID starting with d363b56ab0af60b79dca0b3db46e219444157cac52cd28f1b687686a02aa1289 not found: ID does not exist"
Nov 02 15:35:05.588055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.588012    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:35:05.596033 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.595992    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-de9c4ba4-6fd64c7bb-97dhw"]
Nov 02 15:35:05.707890 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:05.707856    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f814477a-361f-45a2-ba22-c204953004c9" path="/var/lib/kubelet/pods/f814477a-361f-45a2-ba22-c204953004c9/volumes"
Nov 02 15:35:12.227574 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:12.227532    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:35:27.254236 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.254165    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" podUID="e971f64c-b234-42b7-aba2-99ffb6e68532" containerName="main" containerID="cri-o://2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4" gracePeriod=45
Nov 02 15:35:27.511570 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.511509    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:35:27.541545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541506    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541556    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\" (UniqueName: \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541611    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541653    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541724    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\" (UniqueName: \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541775    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\") pod \"e971f64c-b234-42b7-aba2-99ffb6e68532\" (UID: \"e971f64c-b234-42b7-aba2-99ffb6e68532\") "
Nov 02 15:35:27.541983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541826    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:27.541983 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541966    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:27.542158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.541980    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:27.542393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.542360    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.542393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.542386    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.542575 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.542400    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.543331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.543305    4470 generic.go:356] "Generic (PLEG): container finished" podID="e971f64c-b234-42b7-aba2-99ffb6e68532" containerID="2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4" exitCode=0
Nov 02 15:35:27.543446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.543353    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"
Nov 02 15:35:27.543446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.543403    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" event={"ID":"e971f64c-b234-42b7-aba2-99ffb6e68532","Type":"ContainerDied","Data":"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"}
Nov 02 15:35:27.543534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.543465    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x" event={"ID":"e971f64c-b234-42b7-aba2-99ffb6e68532","Type":"ContainerDied","Data":"108cbc572abaf193817a772cdb8f79e823283ecc8aa2db91e05d336ca0fced03"}
Nov 02 15:35:27.543534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.543491    4470 scope.go:122] "RemoveContainer" containerID="2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"
Nov 02 15:35:27.544377 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.544356    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:27.544478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.544452    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:27.544619 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.544554    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t" pod "e971f64c-b234-42b7-aba2-99ffb6e68532" (UID: "e971f64c-b234-42b7-aba2-99ffb6e68532"). InnerVolumeSpecName "kube-api-access-95x7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:27.570675 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.570644    4470 scope.go:122] "RemoveContainer" containerID="2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"
Nov 02 15:35:27.571026 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:27.571008    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4\": container with ID starting with 2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4 not found: ID does not exist" containerID="2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"
Nov 02 15:35:27.571068 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.571038    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4"} err="failed to get container status \"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4\": rpc error: code = NotFound desc = could not find container \"2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4\": container with ID starting with 2e4edef0d3047078192fe1cd0014b09d95185d4b8de23e963600e7c5e4f885a4 not found: ID does not exist"
Nov 02 15:35:27.643688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.643654    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95x7t\" (UniqueName: \"kubernetes.io/projected/e971f64c-b234-42b7-aba2-99ffb6e68532-kube-api-access-95x7t\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.643688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.643685    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e971f64c-b234-42b7-aba2-99ffb6e68532-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.643688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.643695    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e971f64c-b234-42b7-aba2-99ffb6e68532-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:27.959644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.959606    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:35:27.968336 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:27.968305    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-de9c4ba4/e2e-lifecycle-model-de9c4ba4-kserve-86747ddc49-lcw8x"]
Nov 02 15:35:29.706466 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:29.706431    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e971f64c-b234-42b7-aba2-99ffb6e68532" path="/var/lib/kubelet/pods/e971f64c-b234-42b7-aba2-99ffb6e68532/volumes"
Nov 02 15:35:30.921672 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:30.921636    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:35:30.922085 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:30.921850    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" podUID="e900aca8-347a-400f-8743-bb3fdd0988f0" containerName="maas-api" containerID="cri-o://afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127" gracePeriod=30
Nov 02 15:35:31.242688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.242650    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:35:31.309967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.309932    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\") pod \"e900aca8-347a-400f-8743-bb3fdd0988f0\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") "
Nov 02 15:35:31.310172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.309979    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\" (UniqueName: \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\") pod \"e900aca8-347a-400f-8743-bb3fdd0988f0\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") "
Nov 02 15:35:31.310172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.310092    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\") pod \"e900aca8-347a-400f-8743-bb3fdd0988f0\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") "
Nov 02 15:35:31.310172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.310135    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") pod \"e900aca8-347a-400f-8743-bb3fdd0988f0\" (UID: \"e900aca8-347a-400f-8743-bb3fdd0988f0\") "
Nov 02 15:35:31.310527 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.310498    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca" pod "e900aca8-347a-400f-8743-bb3fdd0988f0" (UID: "e900aca8-347a-400f-8743-bb3fdd0988f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:35:31.310762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.310736    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e900aca8-347a-400f-8743-bb3fdd0988f0-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:31.312279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.312252    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs" pod "e900aca8-347a-400f-8743-bb3fdd0988f0" (UID: "e900aca8-347a-400f-8743-bb3fdd0988f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:31.312388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.312300    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw" pod "e900aca8-347a-400f-8743-bb3fdd0988f0" (UID: "e900aca8-347a-400f-8743-bb3fdd0988f0"). InnerVolumeSpecName "kube-api-access-8qsnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:31.312388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.312313    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls" pod "e900aca8-347a-400f-8743-bb3fdd0988f0" (UID: "e900aca8-347a-400f-8743-bb3fdd0988f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:31.412120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.412083    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:31.412120 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.412123    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qsnw\" (UniqueName: \"kubernetes.io/projected/e900aca8-347a-400f-8743-bb3fdd0988f0-kube-api-access-8qsnw\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:31.412398 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.412141    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e900aca8-347a-400f-8743-bb3fdd0988f0-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:31.571667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.571579    4470 generic.go:356] "Generic (PLEG): container finished" podID="e900aca8-347a-400f-8743-bb3fdd0988f0" containerID="afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127" exitCode=0
Nov 02 15:35:31.571667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.571628    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"
Nov 02 15:35:31.571858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.571666    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" event={"ID":"e900aca8-347a-400f-8743-bb3fdd0988f0","Type":"ContainerDied","Data":"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"}
Nov 02 15:35:31.571858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.571727    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv" event={"ID":"e900aca8-347a-400f-8743-bb3fdd0988f0","Type":"ContainerDied","Data":"a67beb48ce872c43424c0344deec9837f14d4f2aa10285d81682060228071f71"}
Nov 02 15:35:31.571858 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.571756    4470 scope.go:122] "RemoveContainer" containerID="afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"
Nov 02 15:35:31.583088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.583070    4470 scope.go:122] "RemoveContainer" containerID="afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"
Nov 02 15:35:31.583396 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:31.583377    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127\": container with ID starting with afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127 not found: ID does not exist" containerID="afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"
Nov 02 15:35:31.583452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.583408    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127"} err="failed to get container status \"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127\": rpc error: code = NotFound desc = could not find container \"afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127\": container with ID starting with afa38608242c68f63199c31a7ba07434b12018dfab6131344f4bf91180016127 not found: ID does not exist"
Nov 02 15:35:31.738872 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.738834    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:35:31.742396 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:31.742362    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-42555d-5cdbf5f995-ffbgv"]
Nov 02 15:35:33.566319 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:33.566283    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:35:33.706482 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:33.706449    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e900aca8-347a-400f-8743-bb3fdd0988f0" path="/var/lib/kubelet/pods/e900aca8-347a-400f-8743-bb3fdd0988f0/volumes"
Nov 02 15:35:34.968647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:34.968612    4470 scope.go:122] "RemoveContainer" containerID="0cbf450775e10abf2556058a25ff6a32da119b666d5e324e0aadf9752ccd3393"
Nov 02 15:35:40.498410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:40.498371    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:35:40.498830 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:40.498637    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" containerID="cri-o://9678ebe81e126d275900111996a37133d7a68922f62765b3294971edbbbb5eed" gracePeriod=30
Nov 02 15:35:45.581253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.581218    4470 patch_prober.go:29] interesting pod/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 02 15:35:45.581631 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.581283    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 02 15:35:45.661594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.661550    4470 generic.go:356] "Generic (PLEG): container finished" podID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerID="9678ebe81e126d275900111996a37133d7a68922f62765b3294971edbbbb5eed" exitCode=0
Nov 02 15:35:45.661771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.661620    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerDied","Data":"9678ebe81e126d275900111996a37133d7a68922f62765b3294971edbbbb5eed"}
Nov 02 15:35:45.661771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.661697    4470 scope.go:122] "RemoveContainer" containerID="545af7fdc79274c2f0f8a2a414a497d2f459cc87e3f4d9f50a1539bc3454acf3"
Nov 02 15:35:45.783856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.783788    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:35:45.912812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.912775    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.912873    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.912914    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.912946    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913005    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913058    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913116    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913228    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913305    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\") pod \"00b64196-9cfa-42fb-8e9c-d6237ba80507\" (UID: \"00b64196-9cfa-42fb-8e9c-d6237ba80507\") "
Nov 02 15:35:45.913436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913343    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:45.913436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913420    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:45.913664 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913641    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:45.913733 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913681    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:35:45.913896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913859    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:45.914007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913912    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:45.914007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913932    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:45.914007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913946    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00b64196-9cfa-42fb-8e9c-d6237ba80507-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:45.914007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.913959    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:45.915568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.915538    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:35:45.915688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.915576    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:35:45.915741 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.915699    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:45.915913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:45.915894    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh" pod "00b64196-9cfa-42fb-8e9c-d6237ba80507" (UID: "00b64196-9cfa-42fb-8e9c-d6237ba80507"). InnerVolumeSpecName "kube-api-access-982nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:46.015505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.015467    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:46.015505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.015508    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00b64196-9cfa-42fb-8e9c-d6237ba80507-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:46.015711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.015524    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:46.015711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.015542    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-982nh\" (UniqueName: \"kubernetes.io/projected/00b64196-9cfa-42fb-8e9c-d6237ba80507-kube-api-access-982nh\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:46.015711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.015558    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00b64196-9cfa-42fb-8e9c-d6237ba80507-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:46.677136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.677103    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"
Nov 02 15:35:46.677136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.677108    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md" event={"ID":"00b64196-9cfa-42fb-8e9c-d6237ba80507","Type":"ContainerDied","Data":"7b3978e3abf3b2c6045c9ba387914924d2e32ee0de58ffae6cba2ad90e00e467"}
Nov 02 15:35:46.677680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.677218    4470 scope.go:122] "RemoveContainer" containerID="9678ebe81e126d275900111996a37133d7a68922f62765b3294971edbbbb5eed"
Nov 02 15:35:46.897579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.897543    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:35:46.904322 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:46.904286    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-42555d-openshift-default-6c8c6d89c-9v4md"]
Nov 02 15:35:47.177216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.177149    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:35:47.177443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.177413    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" podUID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" containerName="maas-api" containerID="cri-o://efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475" gracePeriod=30
Nov 02 15:35:47.513301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.513270    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:35:47.635765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.635729    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") pod \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") "
Nov 02 15:35:47.635965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.635784    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\" (UniqueName: \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\") pod \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") "
Nov 02 15:35:47.635965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.635832    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\") pod \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") "
Nov 02 15:35:47.635965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.635913    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\") pod \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\" (UID: \"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c\") "
Nov 02 15:35:47.636298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.636269    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca" pod "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" (UID: "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:35:47.636597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.636566    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:47.638037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.638008    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs" pod "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" (UID: "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:47.638176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.638049    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h" pod "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" (UID: "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c"). InnerVolumeSpecName "kube-api-access-4zc4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:35:47.638176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.638067    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls" pod "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" (UID: "2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:35:47.685616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.685570    4470 generic.go:356] "Generic (PLEG): container finished" podID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" containerID="efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475" exitCode=0
Nov 02 15:35:47.686116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.685639    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"
Nov 02 15:35:47.686116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.685653    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" event={"ID":"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c","Type":"ContainerDied","Data":"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"}
Nov 02 15:35:47.686116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.685711    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc" event={"ID":"2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c","Type":"ContainerDied","Data":"41aba0d2cbd712ee6cad19653065b93e316c2fc3e877b7df16e58ac3e9337f7e"}
Nov 02 15:35:47.686116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.685734    4470 scope.go:122] "RemoveContainer" containerID="efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"
Nov 02 15:35:47.696544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.696520    4470 scope.go:122] "RemoveContainer" containerID="efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"
Nov 02 15:35:47.696870 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:35:47.696849    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475\": container with ID starting with efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475 not found: ID does not exist" containerID="efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"
Nov 02 15:35:47.696947 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.696878    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475"} err="failed to get container status \"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475\": rpc error: code = NotFound desc = could not find container \"efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475\": container with ID starting with efe27d4658c993bbb6650661edab3aab49dbf0a1bdbd56fa360ec100bc012475 not found: ID does not exist"
Nov 02 15:35:47.707016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.706982    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" path="/var/lib/kubelet/pods/00b64196-9cfa-42fb-8e9c-d6237ba80507/volumes"
Nov 02 15:35:47.737882 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.737803    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:47.737882 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.737832    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:47.737882 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.737845    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zc4h\" (UniqueName: \"kubernetes.io/projected/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c-kube-api-access-4zc4h\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:35:47.852046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.851996    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:35:47.856086 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:47.856051    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-de35cd-56654b74d7-795pc"]
Nov 02 15:35:49.706690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:49.706641    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" path="/var/lib/kubelet/pods/2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c/volumes"
Nov 02 15:35:56.694837 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:56.694796    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:35:56.695304 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:56.695042    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy" containerID="cri-o://f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62" gracePeriod=30
Nov 02 15:35:58.578744 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.578706    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:35:58.579277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579262    4470 cpu_manager.go:414] "containerMap: removing container" podUID="f814477a-361f-45a2-ba22-c204953004c9" containerName="maas-api"
Nov 02 15:35:58.579277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579277    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f814477a-361f-45a2-ba22-c204953004c9" containerName="maas-api"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579292    4470 cpu_manager.go:414] "containerMap: removing container" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579297    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579309    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e971f64c-b234-42b7-aba2-99ffb6e68532" containerName="main"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579314    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e971f64c-b234-42b7-aba2-99ffb6e68532" containerName="main"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579327    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e900aca8-347a-400f-8743-bb3fdd0988f0" containerName="maas-api"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579333    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e900aca8-347a-400f-8743-bb3fdd0988f0" containerName="maas-api"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579343    4470 cpu_manager.go:414] "containerMap: removing container" podUID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" containerName="maas-api"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579347    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" containerName="maas-api"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579361    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579365    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579377    4470 cpu_manager.go:414] "containerMap: removing container" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main"
Nov 02 15:35:58.579389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579382    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579481    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="f814477a-361f-45a2-ba22-c204953004c9" containerName="maas-api"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579491    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579499    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e971f64c-b234-42b7-aba2-99ffb6e68532" containerName="main"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579505    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e900aca8-347a-400f-8743-bb3fdd0988f0" containerName="maas-api"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579512    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ebf7d2f-67c9-44ef-ad3d-071b0fa7744c" containerName="maas-api"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579518    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a915c87-81cc-4937-b9fc-c7d78ff67832" containerName="main"
Nov 02 15:35:58.579755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.579527    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="679279a9-76bb-49aa-a850-04469ea87fcf" containerName="main"
Nov 02 15:35:58.584311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.584290    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.688503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.688407    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4e4683\"/\"test-model-4e4683-kserve-self-signed-certs\""
Nov 02 15:35:58.688503 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.688476    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4e4683\"/\"openshift-service-ca.crt\""
Nov 02 15:35:58.688784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.688539    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4e4683\"/\"kube-root-ca.crt\""
Nov 02 15:35:58.688784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.688650    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4e4683\"/\"default-dockercfg-l2prj\""
Nov 02 15:35:58.692966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.692940    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:35:58.694711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.694680    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.694847 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.694724    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.694914 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.694892    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.695064 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.695044    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz9km\" (UniqueName: \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.695179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.695163    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.695258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.695243    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797148    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797284    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797401 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797389    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797419    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797488    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797546    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797630    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz9km\" (UniqueName: \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797748 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797657    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.797916 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.797887    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.800361 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.800335    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.800685 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.800665    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.858602 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.858518    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz9km\" (UniqueName: \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\") pod \"test-model-4e4683-kserve-7f7fff6995-ntbmd\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") " pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:58.897483 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:58.897448    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:35:59.282420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:59.282390    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:35:59.283960 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:35:59.283934    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cc06f40_392b_4832_a0bf_04313cf2f6de.slice/crio-75b4fad675dcd219e9f2cc3805f8b7f6bd8b05dc7ce3ef5cc98e8b27921c2417 WatchSource:0}: Error finding container 75b4fad675dcd219e9f2cc3805f8b7f6bd8b05dc7ce3ef5cc98e8b27921c2417: Status 404 returned error can't find the container with id 75b4fad675dcd219e9f2cc3805f8b7f6bd8b05dc7ce3ef5cc98e8b27921c2417
Nov 02 15:35:59.774423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:59.774389    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" event={"ID":"1cc06f40-392b-4832-a0bf-04313cf2f6de","Type":"ContainerStarted","Data":"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"}
Nov 02 15:35:59.774841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:59.774431    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" event={"ID":"1cc06f40-392b-4832-a0bf-04313cf2f6de","Type":"ContainerStarted","Data":"75b4fad675dcd219e9f2cc3805f8b7f6bd8b05dc7ce3ef5cc98e8b27921c2417"}
Nov 02 15:35:59.894158 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:59.894093    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" podStartSLOduration=1.894078787 podStartE2EDuration="1.894078787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:35:59.891370762 +0000 UTC m=+2366.329272904" watchObservedRunningTime="2026-11-02 15:35:59.894078787 +0000 UTC m=+2366.331980927"
Nov 02 15:35:59.899316 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:35:59.899285    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:36:01.964946 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:01.964877    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:36:02.040840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.040797    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.040856    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.040889    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.040963    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041009    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041059    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041112    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041148    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041299    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\") pod \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\" (UID: \"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877\") "
Nov 02 15:36:02.041952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041606    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:02.041952 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.041920    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.042389 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.042353    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:02.042495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.042460    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:02.042610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.042578    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:36:02.042822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.042781    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:02.043393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.043368    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:02.043539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.043521    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:36:02.043731 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.043706    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "kube-api-access-jvrv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:02.044219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.044174    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy" pod "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" (UID: "eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:02.143624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143586    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143626    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143642    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143657    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143674    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvrv5\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-kube-api-access-jvrv5\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143690    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143705    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.143821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.143723    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:02.794385 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.794341    4470 generic.go:356] "Generic (PLEG): container finished" podID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerID="f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62" exitCode=0
Nov 02 15:36:02.794571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.794394    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"
Nov 02 15:36:02.794571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.794422    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" event={"ID":"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877","Type":"ContainerDied","Data":"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"}
Nov 02 15:36:02.794571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.794483    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7" event={"ID":"eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877","Type":"ContainerDied","Data":"657e78ebc40a6aa3b7967f8c2fbe9bbd8bafdcaafc5def6a33487631481fbae9"}
Nov 02 15:36:02.794571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.794507    4470 scope.go:122] "RemoveContainer" containerID="f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"
Nov 02 15:36:02.805294 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.805275    4470 scope.go:122] "RemoveContainer" containerID="f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"
Nov 02 15:36:02.805607 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:36:02.805583    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62\": container with ID starting with f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62 not found: ID does not exist" containerID="f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"
Nov 02 15:36:02.805691 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.805633    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62"} err="failed to get container status \"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62\": rpc error: code = NotFound desc = could not find container \"f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62\": container with ID starting with f945005b1af5d2b51424213cbe531055e1e4a1a33d1d2d2d961663f1edbc5d62 not found: ID does not exist"
Nov 02 15:36:02.965197 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:02.965150    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:36:03.014241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:03.014204    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-de35cd-openshift-default-684fbf96fc-2zkp7"]
Nov 02 15:36:03.569635 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:36:03.569593    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:36:03.708571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:03.708538    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" path="/var/lib/kubelet/pods/eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877/volumes"
Nov 02 15:36:08.898123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:08.898082    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:36:08.911589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:08.911562    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:36:09.853090 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:09.853061    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:36:14.925258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:14.925215    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:36:14.925717 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:14.925430    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy" containerID="cri-o://d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6" gracePeriod=30
Nov 02 15:36:14.953873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:14.953840    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:36:14.954152 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:14.954122    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy" containerID="cri-o://3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb" gracePeriod=30
Nov 02 15:36:20.213648 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.213617    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:36:20.245842 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.245815    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:36:20.384148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384112    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.384370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384159    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384222    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384243    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384265    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384325    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384408    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384483    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384523    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384556    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384569    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384593    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384589    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.384657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384627    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384677    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384707    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384747    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384789    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\") pod \"d6a06ba5-14b6-476c-9cee-2d45afa33684\" (UID: \"d6a06ba5-14b6-476c-9cee-2d45afa33684\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384799    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384814    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384825    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384892    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384910    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.385069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.384930    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\") pod \"009fd0fc-b76d-4502-9cfe-0de41af187b6\" (UID: \"009fd0fc-b76d-4502-9cfe-0de41af187b6\") "
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.385113    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.386487    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387107    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387131    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387165    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/009fd0fc-b76d-4502-9cfe-0de41af187b6-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387182    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387223    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387236    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387250    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.387836    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388236    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:36:20.388458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388386    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "kube-api-access-h456l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:20.389134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388672    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.389134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388702    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.389134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388735    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:36:20.389134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.388394    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token" pod "d6a06ba5-14b6-476c-9cee-2d45afa33684" (UID: "d6a06ba5-14b6-476c-9cee-2d45afa33684"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:20.389134 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.389114    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:36:20.389404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.389286    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:20.389461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.389421    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67" pod "009fd0fc-b76d-4502-9cfe-0de41af187b6" (UID: "009fd0fc-b76d-4502-9cfe-0de41af187b6"). InnerVolumeSpecName "kube-api-access-kzf67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488521    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6a06ba5-14b6-476c-9cee-2d45afa33684-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488551    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488562    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzf67\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-kube-api-access-kzf67\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488572    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488583    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/009fd0fc-b76d-4502-9cfe-0de41af187b6-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488593    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488603    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.488604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488612    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6a06ba5-14b6-476c-9cee-2d45afa33684-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.489014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488620    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/009fd0fc-b76d-4502-9cfe-0de41af187b6-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.489014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488630    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h456l\" (UniqueName: \"kubernetes.io/projected/d6a06ba5-14b6-476c-9cee-2d45afa33684-kube-api-access-h456l\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.489014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.488638    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6a06ba5-14b6-476c-9cee-2d45afa33684-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:36:20.907329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.907230    4470 generic.go:356] "Generic (PLEG): container finished" podID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerID="3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb" exitCode=0
Nov 02 15:36:20.907329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.907303    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"
Nov 02 15:36:20.907329 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.907312    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" event={"ID":"d6a06ba5-14b6-476c-9cee-2d45afa33684","Type":"ContainerDied","Data":"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"}
Nov 02 15:36:20.907624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.907378    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k" event={"ID":"d6a06ba5-14b6-476c-9cee-2d45afa33684","Type":"ContainerDied","Data":"95fd9003a16e0bb54f92462ee22d71e16a85ea3e0e2bdd7d76c4d676020decf4"}
Nov 02 15:36:20.907624 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.907414    4470 scope.go:122] "RemoveContainer" containerID="3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"
Nov 02 15:36:20.909072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.909043    4470 generic.go:356] "Generic (PLEG): container finished" podID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerID="d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6" exitCode=0
Nov 02 15:36:20.909225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.909109    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"
Nov 02 15:36:20.909225 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.909129    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" event={"ID":"009fd0fc-b76d-4502-9cfe-0de41af187b6","Type":"ContainerDied","Data":"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"}
Nov 02 15:36:20.909333 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.909309    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt" event={"ID":"009fd0fc-b76d-4502-9cfe-0de41af187b6","Type":"ContainerDied","Data":"4534cfb50d0f4361ae42eba46dd9532867536a9e8e34d2c58ffcd8d0c185e899"}
Nov 02 15:36:20.919149 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.919117    4470 scope.go:122] "RemoveContainer" containerID="3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"
Nov 02 15:36:20.919645 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:36:20.919483    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb\": container with ID starting with 3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb not found: ID does not exist" containerID="3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"
Nov 02 15:36:20.919645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.919513    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb"} err="failed to get container status \"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb\": rpc error: code = NotFound desc = could not find container \"3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb\": container with ID starting with 3c968e82d19947e41739193af6ac916ff34c97ba8eccd753137d5eb35ab3acdb not found: ID does not exist"
Nov 02 15:36:20.919645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.919530    4470 scope.go:122] "RemoveContainer" containerID="d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"
Nov 02 15:36:20.929940 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.929919    4470 scope.go:122] "RemoveContainer" containerID="d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"
Nov 02 15:36:20.930258 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:36:20.930237    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6\": container with ID starting with d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6 not found: ID does not exist" containerID="d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"
Nov 02 15:36:20.930321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:20.930268    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6"} err="failed to get container status \"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6\": rpc error: code = NotFound desc = could not find container \"d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6\": container with ID starting with d1eb4e1535052545a3a6c98fd53fa225ca65389c2ccf921dcaa6afb43e7000a6 not found: ID does not exist"
Nov 02 15:36:21.073296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.073257    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:36:21.122523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.122482    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-dgwmt"]
Nov 02 15:36:21.187412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.187377    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:36:21.192416 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.192389    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-de9c4ba4-openshift-default-854f5f846f-pcc7k"]
Nov 02 15:36:21.706198 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.706153    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" path="/var/lib/kubelet/pods/009fd0fc-b76d-4502-9cfe-0de41af187b6/volumes"
Nov 02 15:36:21.706640 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:21.706625    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" path="/var/lib/kubelet/pods/d6a06ba5-14b6-476c-9cee-2d45afa33684/volumes"
Nov 02 15:36:33.566705 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:36:33.566666    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:36:34.047750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:34.047725    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:36:34.085544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:36:34.085519    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:37:03.565131 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:37:03.565089    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:37:18.435422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.435385    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:37:18.436069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.435637    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" podUID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" containerName="maas-api" containerID="cri-o://5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3" gracePeriod=30
Nov 02 15:37:18.721072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.721044    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:37:18.783740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.783710    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\") pod \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") "
Nov 02 15:37:18.783944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.783932    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\") pod \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") "
Nov 02 15:37:18.784081 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.784063    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\" (UniqueName: \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\") pod \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") "
Nov 02 15:37:18.784148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.784109    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") pod \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\" (UID: \"07ad61b5-e9b8-49c2-91a6-88ef072cd430\") "
Nov 02 15:37:18.784405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.784377    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca" pod "07ad61b5-e9b8-49c2-91a6-88ef072cd430" (UID: "07ad61b5-e9b8-49c2-91a6-88ef072cd430"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:37:18.784943 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.784676    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07ad61b5-e9b8-49c2-91a6-88ef072cd430-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:18.786450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.786426    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs" pod "07ad61b5-e9b8-49c2-91a6-88ef072cd430" (UID: "07ad61b5-e9b8-49c2-91a6-88ef072cd430"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:37:18.786752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.786732    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls" pod "07ad61b5-e9b8-49c2-91a6-88ef072cd430" (UID: "07ad61b5-e9b8-49c2-91a6-88ef072cd430"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:37:18.786752 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.786738    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs" pod "07ad61b5-e9b8-49c2-91a6-88ef072cd430" (UID: "07ad61b5-e9b8-49c2-91a6-88ef072cd430"). InnerVolumeSpecName "kube-api-access-6pjqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:37:18.888243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.887745    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pjqs\" (UniqueName: \"kubernetes.io/projected/07ad61b5-e9b8-49c2-91a6-88ef072cd430-kube-api-access-6pjqs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:18.888243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.887787    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:18.888243 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:18.887801    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07ad61b5-e9b8-49c2-91a6-88ef072cd430-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:19.325215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.321743    4470 generic.go:356] "Generic (PLEG): container finished" podID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" containerID="5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3" exitCode=0
Nov 02 15:37:19.325215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.321937    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" event={"ID":"07ad61b5-e9b8-49c2-91a6-88ef072cd430","Type":"ContainerDied","Data":"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"}
Nov 02 15:37:19.325215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.321995    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl" event={"ID":"07ad61b5-e9b8-49c2-91a6-88ef072cd430","Type":"ContainerDied","Data":"4026b3921e55a765dbf49f23e360ccdc2e07db018203b06a532b2024acb0a0d5"}
Nov 02 15:37:19.325215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.322027    4470 scope.go:122] "RemoveContainer" containerID="5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"
Nov 02 15:37:19.325215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.322183    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"
Nov 02 15:37:19.352383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.352354    4470 scope.go:122] "RemoveContainer" containerID="5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"
Nov 02 15:37:19.352846 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:37:19.352820    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3\": container with ID starting with 5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3 not found: ID does not exist" containerID="5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"
Nov 02 15:37:19.352903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.352859    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3"} err="failed to get container status \"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3\": rpc error: code = NotFound desc = could not find container \"5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3\": container with ID starting with 5ac0917903c4ff1ac425421d28bf2999a59d9038a00644b8da216ca034ab1ab3 not found: ID does not exist"
Nov 02 15:37:19.494133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.494054    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:37:19.543610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.543569    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-db3931-5895bcb5c4-skpgl"]
Nov 02 15:37:19.705549 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:19.705512    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" path="/var/lib/kubelet/pods/07ad61b5-e9b8-49c2-91a6-88ef072cd430/volumes"
Nov 02 15:37:29.180275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:29.180232    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:37:33.565637 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:37:33.565589    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:37:36.224013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:36.223966    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:37:41.224718 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.224662    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" podUID="8fbd675a-602a-43f9-a364-42735927789e" containerName="payload-processing" containerID="cri-o://3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774" gracePeriod=25
Nov 02 15:37:41.488395 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.488333    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:37:41.537619 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.537589    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\") pod \"8fbd675a-602a-43f9-a364-42735927789e\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") "
Nov 02 15:37:41.537772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.537745    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\" (UniqueName: \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\") pod \"8fbd675a-602a-43f9-a364-42735927789e\" (UID: \"8fbd675a-602a-43f9-a364-42735927789e\") "
Nov 02 15:37:41.537997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.537968    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume" pod "8fbd675a-602a-43f9-a364-42735927789e" (UID: "8fbd675a-602a-43f9-a364-42735927789e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:37:41.539987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.539960    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q" pod "8fbd675a-602a-43f9-a364-42735927789e" (UID: "8fbd675a-602a-43f9-a364-42735927789e"). InnerVolumeSpecName "kube-api-access-h5n5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:37:41.582047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.582007    4470 generic.go:356] "Generic (PLEG): container finished" podID="8fbd675a-602a-43f9-a364-42735927789e" containerID="3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774" exitCode=0
Nov 02 15:37:41.582263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.582062    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"
Nov 02 15:37:41.582263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.582093    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" event={"ID":"8fbd675a-602a-43f9-a364-42735927789e","Type":"ContainerDied","Data":"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"}
Nov 02 15:37:41.582263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.582154    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4" event={"ID":"8fbd675a-602a-43f9-a364-42735927789e","Type":"ContainerDied","Data":"c5626cc4f9071dceeef3922b22b97a8e97b946e025050df066b80cc88fd29ff0"}
Nov 02 15:37:41.582263 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.582180    4470 scope.go:122] "RemoveContainer" containerID="3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"
Nov 02 15:37:41.592033 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.592014    4470 scope.go:122] "RemoveContainer" containerID="3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"
Nov 02 15:37:41.592383 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:37:41.592360    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774\": container with ID starting with 3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774 not found: ID does not exist" containerID="3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"
Nov 02 15:37:41.592442 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.592394    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774"} err="failed to get container status \"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774\": rpc error: code = NotFound desc = could not find container \"3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774\": container with ID starting with 3102c81c55d972b8613a9677259b53f813234e38328a88d7069f365a48369774 not found: ID does not exist"
Nov 02 15:37:41.639393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.639357    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fbd675a-602a-43f9-a364-42735927789e-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:41.639393 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.639387    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5n5q\" (UniqueName: \"kubernetes.io/projected/8fbd675a-602a-43f9-a364-42735927789e-kube-api-access-h5n5q\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:41.720253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.720218    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:37:41.723471 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:41.723444    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-c18e79-688b5c688f-f2qn4"]
Nov 02 15:37:43.705576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:43.705541    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fbd675a-602a-43f9-a364-42735927789e" path="/var/lib/kubelet/pods/8fbd675a-602a-43f9-a364-42735927789e/volumes"
Nov 02 15:37:44.201861 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.201825    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main" containerID="cri-o://450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1" gracePeriod=45
Nov 02 15:37:44.469686 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.469614    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:37:44.573228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573153    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573283    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\" (UniqueName: \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573362    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573416    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573465    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573507    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\" (UniqueName: \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\") pod \"dd40061a-9e41-4636-9e48-3deac85661d1\" (UID: \"dd40061a-9e41-4636-9e48-3deac85661d1\") "
Nov 02 15:37:44.573679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573615    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:37:44.573734 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573684    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:37:44.573824 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.573800    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:37:44.574024 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.574003    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.574024 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.574028    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.574226 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.574043    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.575737 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.575709    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:37:44.575838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.575771    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:37:44.575838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.575780    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr" pod "dd40061a-9e41-4636-9e48-3deac85661d1" (UID: "dd40061a-9e41-4636-9e48-3deac85661d1"). InnerVolumeSpecName "kube-api-access-7fvcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:37:44.609571 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.609536    4470 generic.go:356] "Generic (PLEG): container finished" podID="dd40061a-9e41-4636-9e48-3deac85661d1" containerID="450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1" exitCode=0
Nov 02 15:37:44.609775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.609605    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"
Nov 02 15:37:44.609775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.609672    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" event={"ID":"dd40061a-9e41-4636-9e48-3deac85661d1","Type":"ContainerDied","Data":"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"}
Nov 02 15:37:44.609775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.609729    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv" event={"ID":"dd40061a-9e41-4636-9e48-3deac85661d1","Type":"ContainerDied","Data":"980d3ca27c0074b917abe34b3471adaf5d8fe73a9e6666e545146b8af5321e2b"}
Nov 02 15:37:44.609775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.609757    4470 scope.go:122] "RemoveContainer" containerID="450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"
Nov 02 15:37:44.622515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.622333    4470 scope.go:122] "RemoveContainer" containerID="450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"
Nov 02 15:37:44.622840 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:37:44.622804    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1\": container with ID starting with 450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1 not found: ID does not exist" containerID="450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"
Nov 02 15:37:44.622961 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.622852    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1"} err="failed to get container status \"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1\": rpc error: code = NotFound desc = could not find container \"450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1\": container with ID starting with 450ae82582116183d61be80588cc5710692775d6462a7bd3ef3f95cd4c7355f1 not found: ID does not exist"
Nov 02 15:37:44.674647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.674611    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fvcr\" (UniqueName: \"kubernetes.io/projected/dd40061a-9e41-4636-9e48-3deac85661d1-kube-api-access-7fvcr\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.674647 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.674644    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd40061a-9e41-4636-9e48-3deac85661d1-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.674877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.674666    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd40061a-9e41-4636-9e48-3deac85661d1-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:37:44.777596 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.777505    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:37:44.784288 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:44.784256    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-c18e79/distinct-2-w6-c18e79-kserve-695df687f5-kchkv"]
Nov 02 15:37:45.707072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:37:45.707039    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" path="/var/lib/kubelet/pods/dd40061a-9e41-4636-9e48-3deac85661d1/volumes"
Nov 02 15:38:03.566266 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:38:03.566226    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:38:07.901594 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:07.901551    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:38:07.902100 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:07.901830    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerName="discovery" containerID="cri-o://a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841" gracePeriod=30
Nov 02 15:38:08.182723 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.182696    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:38:08.262456 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262421    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262478    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262627    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262785 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262671    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262785 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262742    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262783    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.262896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.262816    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\") pod \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\" (UID: \"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5\") "
Nov 02 15:38:08.263326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.263293    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:38:08.265467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.265436    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:38:08.265644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.265623    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:38:08.266103 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.266074    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:38:08.266207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.266142    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:38:08.266207 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.266155    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:38:08.266362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.266337    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd" pod "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" (UID: "513f0dec-6b25-4ad9-a6bb-5bec7b18abb5"). InnerVolumeSpecName "kube-api-access-9pvwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:38:08.364870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364834    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.364870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364864    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-dns-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.364870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364874    4470 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-csr-ca-configmap\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.365116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364886    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pvwd\" (UniqueName: \"kubernetes.io/projected/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-kube-api-access-9pvwd\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.365116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364895    4470 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-cacerts\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.365116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364904    4470 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-istio-kubeconfig\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.365116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.364912    4470 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5-local-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:38:08.753933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.753892    4470 generic.go:356] "Generic (PLEG): container finished" podID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerID="a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841" exitCode=0
Nov 02 15:38:08.754119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.753944    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"
Nov 02 15:38:08.754119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.753978    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" event={"ID":"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5","Type":"ContainerDied","Data":"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"}
Nov 02 15:38:08.754119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.754040    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx" event={"ID":"513f0dec-6b25-4ad9-a6bb-5bec7b18abb5","Type":"ContainerDied","Data":"81a35feb28d7ae0f1d0a02ecf6cb5eff9a93b17ef7b71b6985dc56f78af0d635"}
Nov 02 15:38:08.754119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.754061    4470 scope.go:122] "RemoveContainer" containerID="a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"
Nov 02 15:38:08.768016 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.767996    4470 scope.go:122] "RemoveContainer" containerID="a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"
Nov 02 15:38:08.768397 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:38:08.768377    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841\": container with ID starting with a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841 not found: ID does not exist" containerID="a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"
Nov 02 15:38:08.768481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.768404    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841"} err="failed to get container status \"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841\": rpc error: code = NotFound desc = could not find container \"a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841\": container with ID starting with a79252a0736068650cb51e51afeca7f9e613d4bc924d68cc958a8d2cdba18841 not found: ID does not exist"
Nov 02 15:38:08.929900 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.929864    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:38:08.978696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:08.978603    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-g5twx"]
Nov 02 15:38:09.705775 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:38:09.705743    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" path="/var/lib/kubelet/pods/513f0dec-6b25-4ad9-a6bb-5bec7b18abb5/volumes"
Nov 02 15:38:33.566260 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:38:33.566221    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:39:03.565784 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:39:03.565743    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:39:33.564858 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:39:33.564816    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:39:34.480163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:34.480125    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:39:49.500874 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.500806    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" podUID="1cc06f40-392b-4832-a0bf-04313cf2f6de" containerName="main" containerID="cri-o://fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375" gracePeriod=45
Nov 02 15:39:49.757121 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.757055    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:39:49.905362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905325    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905543 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905450    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905562    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905607    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\" (UniqueName: \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905727 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905642    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905728    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\") pod \"1cc06f40-392b-4832-a0bf-04313cf2f6de\" (UID: \"1cc06f40-392b-4832-a0bf-04313cf2f6de\") "
Nov 02 15:39:49.905783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905727    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:39:49.905997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.905911    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:39:49.906126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.906071    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:39:49.906244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.906224    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:49.906318 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.906250    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:49.906318 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.906266    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:49.907728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.907701    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:39:49.908248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.908222    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:39:49.908248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:49.908236    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km" pod "1cc06f40-392b-4832-a0bf-04313cf2f6de" (UID: "1cc06f40-392b-4832-a0bf-04313cf2f6de"). InnerVolumeSpecName "kube-api-access-sz9km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:39:50.007250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.007144    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cc06f40-392b-4832-a0bf-04313cf2f6de-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:50.007250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.007172    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sz9km\" (UniqueName: \"kubernetes.io/projected/1cc06f40-392b-4832-a0bf-04313cf2f6de-kube-api-access-sz9km\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:50.007250 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.007183    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cc06f40-392b-4832-a0bf-04313cf2f6de-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:39:50.301958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.301868    4470 generic.go:356] "Generic (PLEG): container finished" podID="1cc06f40-392b-4832-a0bf-04313cf2f6de" containerID="fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375" exitCode=0
Nov 02 15:39:50.301958 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.301919    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"
Nov 02 15:39:50.302160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.301967    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" event={"ID":"1cc06f40-392b-4832-a0bf-04313cf2f6de","Type":"ContainerDied","Data":"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"}
Nov 02 15:39:50.302160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.302008    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd" event={"ID":"1cc06f40-392b-4832-a0bf-04313cf2f6de","Type":"ContainerDied","Data":"75b4fad675dcd219e9f2cc3805f8b7f6bd8b05dc7ce3ef5cc98e8b27921c2417"}
Nov 02 15:39:50.302160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.302025    4470 scope.go:122] "RemoveContainer" containerID="fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"
Nov 02 15:39:50.311912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.311893    4470 scope.go:122] "RemoveContainer" containerID="fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"
Nov 02 15:39:50.312217 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:39:50.312178    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375\": container with ID starting with fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375 not found: ID does not exist" containerID="fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"
Nov 02 15:39:50.312276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.312229    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375"} err="failed to get container status \"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375\": rpc error: code = NotFound desc = could not find container \"fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375\": container with ID starting with fa17a8e494c056479dca5dad6ddcc74cd4247722bae261e3dc70719f72fce375 not found: ID does not exist"
Nov 02 15:39:50.466201 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.466158    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:39:50.468174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:50.468145    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4e4683/test-model-4e4683-kserve-7f7fff6995-ntbmd"]
Nov 02 15:39:51.705327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:39:51.705289    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cc06f40-392b-4832-a0bf-04313cf2f6de" path="/var/lib/kubelet/pods/1cc06f40-392b-4832-a0bf-04313cf2f6de/volumes"
Nov 02 15:40:03.564595 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:03.564553    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:40:25.440095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.440052    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:40:25.440616 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.440290    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" podUID="32e6d9b0-9e98-4839-a167-6486d5f27dae" containerName="maas-api" containerID="cri-o://8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5" gracePeriod=30
Nov 02 15:40:25.704708 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.704638    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:40:25.812379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.812342    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\") pod \"32e6d9b0-9e98-4839-a167-6486d5f27dae\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") "
Nov 02 15:40:25.812581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.812427    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\") pod \"32e6d9b0-9e98-4839-a167-6486d5f27dae\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") "
Nov 02 15:40:25.812581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.812493    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") pod \"32e6d9b0-9e98-4839-a167-6486d5f27dae\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") "
Nov 02 15:40:25.812701 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.812585    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\" (UniqueName: \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\") pod \"32e6d9b0-9e98-4839-a167-6486d5f27dae\" (UID: \"32e6d9b0-9e98-4839-a167-6486d5f27dae\") "
Nov 02 15:40:25.812833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.812806    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca" pod "32e6d9b0-9e98-4839-a167-6486d5f27dae" (UID: "32e6d9b0-9e98-4839-a167-6486d5f27dae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:40:25.813175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.813146    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32e6d9b0-9e98-4839-a167-6486d5f27dae-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:25.815762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.815673    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc" pod "32e6d9b0-9e98-4839-a167-6486d5f27dae" (UID: "32e6d9b0-9e98-4839-a167-6486d5f27dae"). InnerVolumeSpecName "kube-api-access-5q4xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:40:25.835471 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.834297    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls" pod "32e6d9b0-9e98-4839-a167-6486d5f27dae" (UID: "32e6d9b0-9e98-4839-a167-6486d5f27dae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:40:25.840770 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.840688    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs" pod "32e6d9b0-9e98-4839-a167-6486d5f27dae" (UID: "32e6d9b0-9e98-4839-a167-6486d5f27dae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:40:25.914678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.914627    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:25.914678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.914672    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q4xc\" (UniqueName: \"kubernetes.io/projected/32e6d9b0-9e98-4839-a167-6486d5f27dae-kube-api-access-5q4xc\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:25.914678 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:25.914685    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32e6d9b0-9e98-4839-a167-6486d5f27dae-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:26.533179 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.533137    4470 generic.go:356] "Generic (PLEG): container finished" podID="32e6d9b0-9e98-4839-a167-6486d5f27dae" containerID="8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5" exitCode=0
Nov 02 15:40:26.533677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.533225    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"
Nov 02 15:40:26.533677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.533227    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" event={"ID":"32e6d9b0-9e98-4839-a167-6486d5f27dae","Type":"ContainerDied","Data":"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"}
Nov 02 15:40:26.533677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.533317    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr" event={"ID":"32e6d9b0-9e98-4839-a167-6486d5f27dae","Type":"ContainerDied","Data":"f3fe7893995de3216a40417bf37a1cc7a9118ea6691c48781912d20d8cfa0dff"}
Nov 02 15:40:26.533677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.533340    4470 scope.go:122] "RemoveContainer" containerID="8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"
Nov 02 15:40:26.543153 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.543129    4470 scope.go:122] "RemoveContainer" containerID="8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"
Nov 02 15:40:26.543490 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:26.543469    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5\": container with ID starting with 8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5 not found: ID does not exist" containerID="8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"
Nov 02 15:40:26.543544 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.543503    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5"} err="failed to get container status \"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5\": rpc error: code = NotFound desc = could not find container \"8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5\": container with ID starting with 8d0974bb5a29e2fbb36ab4eb0d9a9fa93a93011c4f360452e5d9571b445f32c5 not found: ID does not exist"
Nov 02 15:40:26.697853 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.697814    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:40:26.747612 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:26.747535    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-4e4683-7d6d97d47f-gqjgr"]
Nov 02 15:40:27.705621 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:27.705588    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32e6d9b0-9e98-4839-a167-6486d5f27dae" path="/var/lib/kubelet/pods/32e6d9b0-9e98-4839-a167-6486d5f27dae/volumes"
Nov 02 15:40:33.565814 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:33.565764    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:40:34.975880 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:34.975842    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:40:34.976388 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:34.976049    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy" containerID="cri-o://fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9" gracePeriod=30
Nov 02 15:40:40.237124 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.237092    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:40:40.270884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.270795    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.270884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.270843    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.270895    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271009    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271105 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271088    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271124    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271133    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:40:40.271286 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271184    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271338    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:40:40.271618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271585    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:40:40.271618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271598    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271678    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\") pod \"745c00bf-701e-461a-8558-28f1eb90e28c\" (UID: \"745c00bf-701e-461a-8558-28f1eb90e28c\") "
Nov 02 15:40:40.271818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.271783    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:40:40.272104 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272075    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:40:40.272253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272174    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.272253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272214    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/745c00bf-701e-461a-8558-28f1eb90e28c-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.272253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272232    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.272253 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272247    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.272487 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.272261    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.273484 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.273458    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:40:40.273592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.273551    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "kube-api-access-qcgpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:40:40.273655 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.273632    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:40:40.273709 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.273668    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo" pod "745c00bf-701e-461a-8558-28f1eb90e28c" (UID: "745c00bf-701e-461a-8558-28f1eb90e28c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:40:40.373012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.372971    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.373012 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.373004    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/745c00bf-701e-461a-8558-28f1eb90e28c-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.373276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.373023    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcgpn\" (UniqueName: \"kubernetes.io/projected/745c00bf-701e-461a-8558-28f1eb90e28c-kube-api-access-qcgpn\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.373276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.373041    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/745c00bf-701e-461a-8558-28f1eb90e28c-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:40.609180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.609083    4470 generic.go:356] "Generic (PLEG): container finished" podID="745c00bf-701e-461a-8558-28f1eb90e28c" containerID="fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9" exitCode=0
Nov 02 15:40:40.609180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.609140    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"
Nov 02 15:40:40.609180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.609177    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerDied","Data":"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"}
Nov 02 15:40:40.609449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.609253    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g" event={"ID":"745c00bf-701e-461a-8558-28f1eb90e28c","Type":"ContainerDied","Data":"288013d8bae25d3493f528b60f7296032f5ff9e0e7cd939866ea3a971810822d"}
Nov 02 15:40:40.609449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.609279    4470 scope.go:122] "RemoveContainer" containerID="fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"
Nov 02 15:40:40.619650 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.619628    4470 scope.go:122] "RemoveContainer" containerID="4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"
Nov 02 15:40:40.628855 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.628835    4470 scope.go:122] "RemoveContainer" containerID="fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"
Nov 02 15:40:40.629140 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:40.629120    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9\": container with ID starting with fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9 not found: ID does not exist" containerID="fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"
Nov 02 15:40:40.629251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.629148    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9"} err="failed to get container status \"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9\": rpc error: code = NotFound desc = could not find container \"fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9\": container with ID starting with fb6e7ea89d68aa7e7f4aa8772df594c8d669efe0947e3acc83663076c53c90d9 not found: ID does not exist"
Nov 02 15:40:40.629251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.629165    4470 scope.go:122] "RemoveContainer" containerID="4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"
Nov 02 15:40:40.629450 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:40.629430    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e\": container with ID starting with 4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e not found: ID does not exist" containerID="4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"
Nov 02 15:40:40.629492 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.629457    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e"} err="failed to get container status \"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e\": rpc error: code = NotFound desc = could not find container \"4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e\": container with ID starting with 4333808d0f7ea5fa49a2d0cd90fbde0fd31eb859b2471408763e2e2aacb6700e not found: ID does not exist"
Nov 02 15:40:40.782836 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.782801    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:40:40.794302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:40.794274    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4e4683-openshift-default-5d7577cbc7-qb54g"]
Nov 02 15:40:41.706629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:41.706594    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" path="/var/lib/kubelet/pods/745c00bf-701e-461a-8558-28f1eb90e28c/volumes"
Nov 02 15:40:46.878814 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:46.878710    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:40:46.879341 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:46.879042    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" podUID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" containerName="maas-api" containerID="cri-o://f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41" gracePeriod=30
Nov 02 15:40:47.150291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.150261    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:40:47.254551 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.254509    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\") pod \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") "
Nov 02 15:40:47.254803 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.254750    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\") pod \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") "
Nov 02 15:40:47.254897 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.254844    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") pod \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") "
Nov 02 15:40:47.254957 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.254918    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\" (UniqueName: \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\") pod \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\" (UID: \"79bdf7f0-47cb-446e-aa6a-4faea8a71a49\") "
Nov 02 15:40:47.256276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.256063    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca" pod "79bdf7f0-47cb-446e-aa6a-4faea8a71a49" (UID: "79bdf7f0-47cb-446e-aa6a-4faea8a71a49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:40:47.258537 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.258505    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs" pod "79bdf7f0-47cb-446e-aa6a-4faea8a71a49" (UID: "79bdf7f0-47cb-446e-aa6a-4faea8a71a49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:40:47.260641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.260616    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr" pod "79bdf7f0-47cb-446e-aa6a-4faea8a71a49" (UID: "79bdf7f0-47cb-446e-aa6a-4faea8a71a49"). InnerVolumeSpecName "kube-api-access-5b6dr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:40:47.273771 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.273702    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls" pod "79bdf7f0-47cb-446e-aa6a-4faea8a71a49" (UID: "79bdf7f0-47cb-446e-aa6a-4faea8a71a49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:40:47.356671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.356632    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:47.356671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.356669    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:47.356671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.356684    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b6dr\" (UniqueName: \"kubernetes.io/projected/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-kube-api-access-5b6dr\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:47.356936 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.356698    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79bdf7f0-47cb-446e-aa6a-4faea8a71a49-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:40:47.647981 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.647938    4470 generic.go:356] "Generic (PLEG): container finished" podID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" containerID="f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41" exitCode=0
Nov 02 15:40:47.648170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.647991    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"
Nov 02 15:40:47.648170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.648030    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" event={"ID":"79bdf7f0-47cb-446e-aa6a-4faea8a71a49","Type":"ContainerDied","Data":"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"}
Nov 02 15:40:47.648170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.648099    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj" event={"ID":"79bdf7f0-47cb-446e-aa6a-4faea8a71a49","Type":"ContainerDied","Data":"860be9ed1bd9fdc651f8864931fe95e2a4cba60520ccbcbad3fa2ebe2b432d90"}
Nov 02 15:40:47.648170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.648127    4470 scope.go:122] "RemoveContainer" containerID="f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"
Nov 02 15:40:47.658532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.658509    4470 scope.go:122] "RemoveContainer" containerID="f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"
Nov 02 15:40:47.658838 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:40:47.658818    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41\": container with ID starting with f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41 not found: ID does not exist" containerID="f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"
Nov 02 15:40:47.658889 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.658849    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41"} err="failed to get container status \"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41\": rpc error: code = NotFound desc = could not find container \"f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41\": container with ID starting with f2a860567fd2395e0fc160ed7eff23636f976723b15bfc4b33e81bd8120d5f41 not found: ID does not exist"
Nov 02 15:40:47.813203 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.813143    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:40:47.817956 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:47.817924    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-895ac8-8566876687-lvgnj"]
Nov 02 15:40:49.705261 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:40:49.705227    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" path="/var/lib/kubelet/pods/79bdf7f0-47cb-446e-aa6a-4faea8a71a49/volumes"
Nov 02 15:41:03.564458 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:03.564413    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:41:21.948273 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:21.948235    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:41:22.053915 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:22.053876    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:41:22.800765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:22.800729    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:41:22.800997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:22.800973    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" podUID="79610468-a6cf-4732-9531-d4d430701010" containerName="maas-api" containerID="cri-o://316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778" gracePeriod=30
Nov 02 15:41:23.079244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.079215    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:41:23.235018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.234989    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\") pod \"79610468-a6cf-4732-9531-d4d430701010\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") "
Nov 02 15:41:23.235246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.235034    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\" (UniqueName: \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\") pod \"79610468-a6cf-4732-9531-d4d430701010\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") "
Nov 02 15:41:23.235246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.235143    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\") pod \"79610468-a6cf-4732-9531-d4d430701010\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") "
Nov 02 15:41:23.235246 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.235235    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") pod \"79610468-a6cf-4732-9531-d4d430701010\" (UID: \"79610468-a6cf-4732-9531-d4d430701010\") "
Nov 02 15:41:23.235450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.235416    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca" pod "79610468-a6cf-4732-9531-d4d430701010" (UID: "79610468-a6cf-4732-9531-d4d430701010"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:23.235711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.235680    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79610468-a6cf-4732-9531-d4d430701010-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:23.237429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.237401    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs" pod "79610468-a6cf-4732-9531-d4d430701010" (UID: "79610468-a6cf-4732-9531-d4d430701010"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:41:23.237429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.237418    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss" pod "79610468-a6cf-4732-9531-d4d430701010" (UID: "79610468-a6cf-4732-9531-d4d430701010"). InnerVolumeSpecName "kube-api-access-cwdss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:23.237614 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.237438    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls" pod "79610468-a6cf-4732-9531-d4d430701010" (UID: "79610468-a6cf-4732-9531-d4d430701010"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:41:23.337368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.337270    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwdss\" (UniqueName: \"kubernetes.io/projected/79610468-a6cf-4732-9531-d4d430701010-kube-api-access-cwdss\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:23.337368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.337299    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:23.337368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.337311    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79610468-a6cf-4732-9531-d4d430701010-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:23.847289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.847250    4470 generic.go:356] "Generic (PLEG): container finished" podID="79610468-a6cf-4732-9531-d4d430701010" containerID="316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778" exitCode=0
Nov 02 15:41:23.847506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.847334    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"
Nov 02 15:41:23.847506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.847336    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" event={"ID":"79610468-a6cf-4732-9531-d4d430701010","Type":"ContainerDied","Data":"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"}
Nov 02 15:41:23.847506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.847401    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx" event={"ID":"79610468-a6cf-4732-9531-d4d430701010","Type":"ContainerDied","Data":"d43a4080390e554bdedaeb6b8732e44e9e09b4f15ec3e539dee88ea620a0f77a"}
Nov 02 15:41:23.847506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.847424    4470 scope.go:122] "RemoveContainer" containerID="316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"
Nov 02 15:41:23.856637 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.856619    4470 scope.go:122] "RemoveContainer" containerID="316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"
Nov 02 15:41:23.856899 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:23.856878    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778\": container with ID starting with 316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778 not found: ID does not exist" containerID="316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"
Nov 02 15:41:23.856959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:23.856907    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778"} err="failed to get container status \"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778\": rpc error: code = NotFound desc = could not find container \"316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778\": container with ID starting with 316bc5ddc817013220673191192aea1b06bf37fb01f2270411ee5261a928f778 not found: ID does not exist"
Nov 02 15:41:24.014280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:24.014169    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:41:24.016690 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:24.016664    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-fd1339-6c76fd6dbb-9gcgx"]
Nov 02 15:41:25.704838 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:25.704794    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79610468-a6cf-4732-9531-d4d430701010" path="/var/lib/kubelet/pods/79610468-a6cf-4732-9531-d4d430701010/volumes"
Nov 02 15:41:26.949050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:26.949007    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" podUID="d9fe728c-4705-4c04-beca-56bdcb00d15e" containerName="payload-processing" containerID="cri-o://789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b" gracePeriod=25
Nov 02 15:41:27.054990 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.054927    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" podUID="034816b3-6329-41cf-a182-19a9c91a2139" containerName="payload-pre-processing" containerID="cri-o://cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97" gracePeriod=25
Nov 02 15:41:27.208615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.208538    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:41:27.324302 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.324276    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:41:27.380096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.380062    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\") pod \"d9fe728c-4705-4c04-beca-56bdcb00d15e\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") "
Nov 02 15:41:27.380356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.380145    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\" (UniqueName: \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\") pod \"d9fe728c-4705-4c04-beca-56bdcb00d15e\" (UID: \"d9fe728c-4705-4c04-beca-56bdcb00d15e\") "
Nov 02 15:41:27.380515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.380482    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume" pod "d9fe728c-4705-4c04-beca-56bdcb00d15e" (UID: "d9fe728c-4705-4c04-beca-56bdcb00d15e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:27.382435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.382406    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s" pod "d9fe728c-4705-4c04-beca-56bdcb00d15e" (UID: "d9fe728c-4705-4c04-beca-56bdcb00d15e"). InnerVolumeSpecName "kube-api-access-tcs7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:27.481301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.481211    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\" (UniqueName: \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\") pod \"034816b3-6329-41cf-a182-19a9c91a2139\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") "
Nov 02 15:41:27.481472 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.481424    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\") pod \"034816b3-6329-41cf-a182-19a9c91a2139\" (UID: \"034816b3-6329-41cf-a182-19a9c91a2139\") "
Nov 02 15:41:27.481802 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.481771    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume" pod "034816b3-6329-41cf-a182-19a9c91a2139" (UID: "034816b3-6329-41cf-a182-19a9c91a2139"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:27.481802 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.481792    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9fe728c-4705-4c04-beca-56bdcb00d15e-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:27.481967 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.481813    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs7s\" (UniqueName: \"kubernetes.io/projected/d9fe728c-4705-4c04-beca-56bdcb00d15e-kube-api-access-tcs7s\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:27.483577 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.483555    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p" pod "034816b3-6329-41cf-a182-19a9c91a2139" (UID: "034816b3-6329-41cf-a182-19a9c91a2139"). InnerVolumeSpecName "kube-api-access-wzt7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:27.583276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.583232    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034816b3-6329-41cf-a182-19a9c91a2139-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:27.583276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.583271    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzt7p\" (UniqueName: \"kubernetes.io/projected/034816b3-6329-41cf-a182-19a9c91a2139-kube-api-access-wzt7p\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:27.869657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.869564    4470 generic.go:356] "Generic (PLEG): container finished" podID="034816b3-6329-41cf-a182-19a9c91a2139" containerID="cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97" exitCode=0
Nov 02 15:41:27.869657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.869614    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"
Nov 02 15:41:27.869657 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.869646    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" event={"ID":"034816b3-6329-41cf-a182-19a9c91a2139","Type":"ContainerDied","Data":"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"}
Nov 02 15:41:27.869959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.869697    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr" event={"ID":"034816b3-6329-41cf-a182-19a9c91a2139","Type":"ContainerDied","Data":"3b10e46127b0deeb4b61abbc5a26189cd0e3cded214fe4661d0c8892747ef9f4"}
Nov 02 15:41:27.869959 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.869724    4470 scope.go:122] "RemoveContainer" containerID="cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"
Nov 02 15:41:27.871360 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.871336    4470 generic.go:356] "Generic (PLEG): container finished" podID="d9fe728c-4705-4c04-beca-56bdcb00d15e" containerID="789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b" exitCode=0
Nov 02 15:41:27.871500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.871385    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" event={"ID":"d9fe728c-4705-4c04-beca-56bdcb00d15e","Type":"ContainerDied","Data":"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"}
Nov 02 15:41:27.871500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.871389    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"
Nov 02 15:41:27.871500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.871413    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc" event={"ID":"d9fe728c-4705-4c04-beca-56bdcb00d15e","Type":"ContainerDied","Data":"fbc955193957129753f526a728b61ee92bc1d918c5587cb9661a1e224b98dd7b"}
Nov 02 15:41:27.879330 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.879308    4470 scope.go:122] "RemoveContainer" containerID="cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"
Nov 02 15:41:27.879632 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:27.879601    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97\": container with ID starting with cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97 not found: ID does not exist" containerID="cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"
Nov 02 15:41:27.879729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.879640    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97"} err="failed to get container status \"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97\": rpc error: code = NotFound desc = could not find container \"cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97\": container with ID starting with cd509387bf40a86b9eeb6432b69c2cac3d0f9faf4c802d80d6236d77d0dc0a97 not found: ID does not exist"
Nov 02 15:41:27.879729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.879658    4470 scope.go:122] "RemoveContainer" containerID="789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"
Nov 02 15:41:27.888044 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.888023    4470 scope.go:122] "RemoveContainer" containerID="789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"
Nov 02 15:41:27.888372 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:27.888344    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b\": container with ID starting with 789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b not found: ID does not exist" containerID="789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"
Nov 02 15:41:27.888472 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:27.888382    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b"} err="failed to get container status \"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b\": rpc error: code = NotFound desc = could not find container \"789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b\": container with ID starting with 789a70faae41c6b2f646fb9f975b54fb2692e1d1da1f47f4f01e2cd5459fec1b not found: ID does not exist"
Nov 02 15:41:28.035653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:28.035620    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:41:28.085279 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:28.085240    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-fd1339-79fd7c96dc-ktxwr"]
Nov 02 15:41:28.147180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:28.147093    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:41:28.149826 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:28.149800    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-fd1339-7456979db9-ngwcc"]
Nov 02 15:41:29.705682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:29.705648    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="034816b3-6329-41cf-a182-19a9c91a2139" path="/var/lib/kubelet/pods/034816b3-6329-41cf-a182-19a9c91a2139/volumes"
Nov 02 15:41:29.706094 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:29.706024    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9fe728c-4705-4c04-beca-56bdcb00d15e" path="/var/lib/kubelet/pods/d9fe728c-4705-4c04-beca-56bdcb00d15e/volumes"
Nov 02 15:41:33.565221 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:33.565164    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:41:34.080618 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:34.080590    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:41:34.120282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:34.120253    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:41:34.645568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:34.645526    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:41:34.646147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:34.645740    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy" containerID="cri-o://b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7" gracePeriod=30
Nov 02 15:41:39.908857 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.908820    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:41:39.936548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.936513    4470 generic.go:356] "Generic (PLEG): container finished" podID="33eaeb67-0da1-4614-b321-520f16cb8607" containerID="b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7" exitCode=0
Nov 02 15:41:39.936739 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.936580    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"
Nov 02 15:41:39.936739 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.936601    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerDied","Data":"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"}
Nov 02 15:41:39.936739 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.936664    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999" event={"ID":"33eaeb67-0da1-4614-b321-520f16cb8607","Type":"ContainerDied","Data":"191a0726661db43fceeb0de5d0a8d64e44cab2e9dbd5ccbcf71c89c1588699e3"}
Nov 02 15:41:39.936739 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.936687    4470 scope.go:122] "RemoveContainer" containerID="b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"
Nov 02 15:41:39.945660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.945642    4470 scope.go:122] "RemoveContainer" containerID="9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"
Nov 02 15:41:39.954216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.954179    4470 scope.go:122] "RemoveContainer" containerID="b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"
Nov 02 15:41:39.954473 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:39.954455    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7\": container with ID starting with b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7 not found: ID does not exist" containerID="b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"
Nov 02 15:41:39.954534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.954483    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7"} err="failed to get container status \"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7\": rpc error: code = NotFound desc = could not find container \"b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7\": container with ID starting with b79e568d3b9e9c4493b8283dbd6431375093e9f60138bb82fb8bb740bea87ba7 not found: ID does not exist"
Nov 02 15:41:39.954534 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.954503    4470 scope.go:122] "RemoveContainer" containerID="9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"
Nov 02 15:41:39.954750 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:39.954725    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e\": container with ID starting with 9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e not found: ID does not exist" containerID="9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"
Nov 02 15:41:39.954800 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:39.954749    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e"} err="failed to get container status \"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e\": rpc error: code = NotFound desc = could not find container \"9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e\": container with ID starting with 9383dd59313b7e545bdd74f27634c256a6c959db9a389eb649990c1c24b0154e not found: ID does not exist"
Nov 02 15:41:40.009976 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.009941    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010005    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010046    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010119    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010165    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010238    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010269    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010303    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010323    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:40.010420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010374    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\") pod \"33eaeb67-0da1-4614-b321-520f16cb8607\" (UID: \"33eaeb67-0da1-4614-b321-520f16cb8607\") "
Nov 02 15:41:40.010767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010596    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:40.010767 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010632    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:40.010933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010835    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.010933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010843    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:40.010933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010858    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.010933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.010875    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.011198 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.011074    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:40.012586 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.012548    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:40.012876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.012850    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:40.012876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.012871    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "kube-api-access-m6ssw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:40.013219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.013178    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo" pod "33eaeb67-0da1-4614-b321-520f16cb8607" (UID: "33eaeb67-0da1-4614-b321-520f16cb8607"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:41:40.111660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111624    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.111660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111657    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33eaeb67-0da1-4614-b321-520f16cb8607-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.111660 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111666    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.111891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111678    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6ssw\" (UniqueName: \"kubernetes.io/projected/33eaeb67-0da1-4614-b321-520f16cb8607-kube-api-access-m6ssw\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.111891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111693    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33eaeb67-0da1-4614-b321-520f16cb8607-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.111891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.111701    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33eaeb67-0da1-4614-b321-520f16cb8607-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:40.131088 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.131047    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:41:40.373125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.373086    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:41:40.382682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:40.382640    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-fd1339-openshift-default-75f7565c8c-xl999"]
Nov 02 15:41:41.711355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:41.711321    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" path="/var/lib/kubelet/pods/33eaeb67-0da1-4614-b321-520f16cb8607/volumes"
Nov 02 15:41:47.220391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:47.220350    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:41:47.324873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:47.324837    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:41:49.424254 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424209    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"]
Nov 02 15:41:49.424883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424859    4470 cpu_manager.go:414] "containerMap: removing container" podUID="034816b3-6329-41cf-a182-19a9c91a2139" containerName="payload-pre-processing"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424885    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="034816b3-6329-41cf-a182-19a9c91a2139" containerName="payload-pre-processing"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424906    4470 cpu_manager.go:414] "containerMap: removing container" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424914    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424933    4470 cpu_manager.go:414] "containerMap: removing container" podUID="32e6d9b0-9e98-4839-a167-6486d5f27dae" containerName="maas-api"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424942    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32e6d9b0-9e98-4839-a167-6486d5f27dae" containerName="maas-api"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424956    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d9fe728c-4705-4c04-beca-56bdcb00d15e" containerName="payload-processing"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424963    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9fe728c-4705-4c04-beca-56bdcb00d15e" containerName="payload-processing"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424975    4470 cpu_manager.go:414] "containerMap: removing container" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424982    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:41:49.424991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424992    4470 cpu_manager.go:414] "containerMap: removing container" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.424999    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425020    4470 cpu_manager.go:414] "containerMap: removing container" podUID="8fbd675a-602a-43f9-a364-42735927789e" containerName="payload-processing"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425027    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fbd675a-602a-43f9-a364-42735927789e" containerName="payload-processing"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425040    4470 cpu_manager.go:414] "containerMap: removing container" podUID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425048    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425058    4470 cpu_manager.go:414] "containerMap: removing container" podUID="79610468-a6cf-4732-9531-d4d430701010" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425067    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79610468-a6cf-4732-9531-d4d430701010" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425082    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425089    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425108    4470 cpu_manager.go:414] "containerMap: removing container" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425116    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425130    4470 cpu_manager.go:414] "containerMap: removing container" podUID="1cc06f40-392b-4832-a0bf-04313cf2f6de" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425137    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cc06f40-392b-4832-a0bf-04313cf2f6de" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425151    4470 cpu_manager.go:414] "containerMap: removing container" podUID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425157    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425170    4470 cpu_manager.go:414] "containerMap: removing container" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425178    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425231    4470 cpu_manager.go:414] "containerMap: removing container" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerName="discovery"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425238    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerName="discovery"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425255    4470 cpu_manager.go:414] "containerMap: removing container" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425262    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425273    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425280    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425394    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425406    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd40061a-9e41-4636-9e48-3deac85661d1" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425416    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="034816b3-6329-41cf-a182-19a9c91a2139" containerName="payload-pre-processing"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425429    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425438    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="32e6d9b0-9e98-4839-a167-6486d5f27dae" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425447    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9fe728c-4705-4c04-beca-56bdcb00d15e" containerName="payload-processing"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425457    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="00b64196-9cfa-42fb-8e9c-d6237ba80507" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425467    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="009fd0fc-b76d-4502-9cfe-0de41af187b6" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425477    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fbd675a-602a-43f9-a364-42735927789e" containerName="payload-processing"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425486    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="07ad61b5-e9b8-49c2-91a6-88ef072cd430" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425496    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="79610468-a6cf-4732-9531-d4d430701010" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425506    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6a06ba5-14b6-476c-9cee-2d45afa33684" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425517    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb5fa8dc-6aea-427b-8cb7-cd39b8f4e877" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425527    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cc06f40-392b-4832-a0bf-04313cf2f6de" containerName="main"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425537    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="79bdf7f0-47cb-446e-aa6a-4faea8a71a49" containerName="maas-api"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425547    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="33eaeb67-0da1-4614-b321-520f16cb8607" containerName="istio-proxy"
Nov 02 15:41:49.425546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.425557    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="513f0dec-6b25-4ad9-a6bb-5bec7b18abb5" containerName="discovery"
Nov 02 15:41:49.430978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.430951    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.488222 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.488171    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"]
Nov 02 15:41:49.514670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.514617    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.514854 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.514743    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2zhz\" (UniqueName: \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.514903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.514892    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.534929 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.534885    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 15:41:49.535129 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.534952    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pzqc\""
Nov 02 15:41:49.615923 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.615884    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.616143 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.615931    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2zhz\" (UniqueName: \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.616143 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.616017    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.616749 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.616728    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.618457 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.618432    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.677517 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.677429    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2zhz\" (UniqueName: \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\") pod \"maas-api-revoke-sub-76eaf9b0-2f26p\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:49.745736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:49.745699    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:50.127585 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:50.127547    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"]
Nov 02 15:41:50.131016 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:41:50.130981    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce49499e_0667_44c0_813e_8fd7db8f4e33.slice/crio-619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431 WatchSource:0}: Error finding container 619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431: Status 404 returned error can't find the container with id 619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431
Nov 02 15:41:50.993964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:50.993928    4470 generic.go:356] "Generic (PLEG): container finished" podID="ce49499e-0667-44c0-813e-8fd7db8f4e33" containerID="c5bf51dd5d74b00bf0538cd48e4dce87f58dcb733996d273fd42be380f962b00" exitCode=0
Nov 02 15:41:50.994452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:50.993989    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p" event={"ID":"ce49499e-0667-44c0-813e-8fd7db8f4e33","Type":"ContainerDied","Data":"c5bf51dd5d74b00bf0538cd48e4dce87f58dcb733996d273fd42be380f962b00"}
Nov 02 15:41:50.994452 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:50.994029    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p" event={"ID":"ce49499e-0667-44c0-813e-8fd7db8f4e33","Type":"ContainerStarted","Data":"619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431"}
Nov 02 15:41:52.139424 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.139396    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:52.221144 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.221100    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" podUID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" containerName="payload-processing" containerID="cri-o://72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d" gracePeriod=25
Nov 02 15:41:52.246884 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.246850    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\" (UniqueName: \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\") pod \"ce49499e-0667-44c0-813e-8fd7db8f4e33\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") "
Nov 02 15:41:52.247051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.246899    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\") pod \"ce49499e-0667-44c0-813e-8fd7db8f4e33\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") "
Nov 02 15:41:52.247051 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.246998    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\") pod \"ce49499e-0667-44c0-813e-8fd7db8f4e33\" (UID: \"ce49499e-0667-44c0-813e-8fd7db8f4e33\") "
Nov 02 15:41:52.247383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.247360    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca" pod "ce49499e-0667-44c0-813e-8fd7db8f4e33" (UID: "ce49499e-0667-44c0-813e-8fd7db8f4e33"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:52.248990 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.248972    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca" pod "ce49499e-0667-44c0-813e-8fd7db8f4e33" (UID: "ce49499e-0667-44c0-813e-8fd7db8f4e33"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:41:52.249590 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.249535    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz" pod "ce49499e-0667-44c0-813e-8fd7db8f4e33" (UID: "ce49499e-0667-44c0-813e-8fd7db8f4e33"). InnerVolumeSpecName "kube-api-access-q2zhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:52.325480 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.325412    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" podUID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" containerName="payload-pre-processing" containerID="cri-o://163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d" gracePeriod=25
Nov 02 15:41:52.348065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.348032    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2zhz\" (UniqueName: \"kubernetes.io/projected/ce49499e-0667-44c0-813e-8fd7db8f4e33-kube-api-access-q2zhz\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.348065 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.348066    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-xks-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.348276 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.348081    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ce49499e-0667-44c0-813e-8fd7db8f4e33-maas-api-openshift-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.459750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.459725    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:41:52.549928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.549891    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\" (UniqueName: \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\") pod \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") "
Nov 02 15:41:52.550136 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.549943    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\") pod \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\" (UID: \"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6\") "
Nov 02 15:41:52.550451 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.550421    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume" pod "d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" (UID: "d4da8b94-0bcf-4788-b4a8-dde8ebcdede6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:52.552783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.552761    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk" pod "d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" (UID: "d4da8b94-0bcf-4788-b4a8-dde8ebcdede6"). InnerVolumeSpecName "kube-api-access-qb5vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:52.572747 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.572723    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:41:52.651682 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.651647    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\") pod \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") "
Nov 02 15:41:52.651870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.651747    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\" (UniqueName: \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\") pod \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\" (UID: \"4306b86e-471a-4a5d-bdcd-3b679571fcb8\") "
Nov 02 15:41:52.652000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.651989    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qb5vk\" (UniqueName: \"kubernetes.io/projected/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-kube-api-access-qb5vk\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.652039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.652002    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.652084 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.652061    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume" pod "4306b86e-471a-4a5d-bdcd-3b679571fcb8" (UID: "4306b86e-471a-4a5d-bdcd-3b679571fcb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:41:52.653805 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.653786    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s" pod "4306b86e-471a-4a5d-bdcd-3b679571fcb8" (UID: "4306b86e-471a-4a5d-bdcd-3b679571fcb8"). InnerVolumeSpecName "kube-api-access-fkn7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:52.753303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.753218    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4306b86e-471a-4a5d-bdcd-3b679571fcb8-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:52.753303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:52.753249    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkn7s\" (UniqueName: \"kubernetes.io/projected/4306b86e-471a-4a5d-bdcd-3b679571fcb8-kube-api-access-fkn7s\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:53.005552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.005470    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"
Nov 02 15:41:53.005552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.005505    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p" event={"ID":"ce49499e-0667-44c0-813e-8fd7db8f4e33","Type":"ContainerDied","Data":"619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431"}
Nov 02 15:41:53.005761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.005566    4470 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="619dc02d9f4e08333577d4dab4fcbb8bc9765dcd4d2a1a2647794ed9016dc431"
Nov 02 15:41:53.006899 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.006871    4470 generic.go:356] "Generic (PLEG): container finished" podID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" containerID="163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d" exitCode=0
Nov 02 15:41:53.007019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.006907    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"
Nov 02 15:41:53.007019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.006951    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" event={"ID":"4306b86e-471a-4a5d-bdcd-3b679571fcb8","Type":"ContainerDied","Data":"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"}
Nov 02 15:41:53.007019 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.006994    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz" event={"ID":"4306b86e-471a-4a5d-bdcd-3b679571fcb8","Type":"ContainerDied","Data":"5afad91e2ea11e0b9c21032ab61f25f674c700b4f27f15696b63e80745da1a1b"}
Nov 02 15:41:53.007165 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.007022    4470 scope.go:122] "RemoveContainer" containerID="163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"
Nov 02 15:41:53.008486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.008467    4470 generic.go:356] "Generic (PLEG): container finished" podID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" containerID="72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d" exitCode=0
Nov 02 15:41:53.008566 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.008505    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" event={"ID":"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6","Type":"ContainerDied","Data":"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"}
Nov 02 15:41:53.008566 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.008531    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5" event={"ID":"d4da8b94-0bcf-4788-b4a8-dde8ebcdede6","Type":"ContainerDied","Data":"a5754711995ba5647e93968fc19d2c8f86a1f38e5dd3307de8424209dbefb5af"}
Nov 02 15:41:53.008680 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.008570    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"
Nov 02 15:41:53.016804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.016731    4470 scope.go:122] "RemoveContainer" containerID="163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"
Nov 02 15:41:53.017096 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:53.017068    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d\": container with ID starting with 163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d not found: ID does not exist" containerID="163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"
Nov 02 15:41:53.017252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.017106    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d"} err="failed to get container status \"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d\": rpc error: code = NotFound desc = could not find container \"163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d\": container with ID starting with 163d48f83170420b145cb4837c16b1abb61cf1e699e17f83cd1ca441abe30f0d not found: ID does not exist"
Nov 02 15:41:53.017252 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.017130    4470 scope.go:122] "RemoveContainer" containerID="72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"
Nov 02 15:41:53.025755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.025737    4470 scope.go:122] "RemoveContainer" containerID="72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"
Nov 02 15:41:53.026040 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:53.026023    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d\": container with ID starting with 72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d not found: ID does not exist" containerID="72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"
Nov 02 15:41:53.026085 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.026048    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d"} err="failed to get container status \"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d\": rpc error: code = NotFound desc = could not find container \"72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d\": container with ID starting with 72759d8d79425fdcd7fddd3c4a72d85c6ee14a7b53eb6165d0b77fce09ce807d not found: ID does not exist"
Nov 02 15:41:53.224287 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.224256    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:41:53.229898 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.229869    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-019efe-997bbb677-sd9j5"]
Nov 02 15:41:53.342392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.342298    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:41:53.349311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.349277    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-019efe-8544574bbc-652gz"]
Nov 02 15:41:53.705003 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.704970    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" path="/var/lib/kubelet/pods/4306b86e-471a-4a5d-bdcd-3b679571fcb8/volumes"
Nov 02 15:41:53.705422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:53.705407    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" path="/var/lib/kubelet/pods/d4da8b94-0bcf-4788-b4a8-dde8ebcdede6/volumes"
Nov 02 15:41:55.152346 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.152283    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" containerID="cri-o://111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e" gracePeriod=45
Nov 02 15:41:55.262496 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.262456    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": dial tcp 10.133.0.59:8000: connect: connection refused"
Nov 02 15:41:55.409340 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.409266    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:41:55.580543 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580509    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\" (UniqueName: \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.580750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580626    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.580750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580669    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.580750 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580720    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.580912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580754    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.580912 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580853    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\" (UniqueName: \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\") pod \"297573e4-27ee-460d-b847-6ed4f8c45c6b\" (UID: \"297573e4-27ee-460d-b847-6ed4f8c45c6b\") "
Nov 02 15:41:55.581037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.580961    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:55.581037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.581002    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:55.581037 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.581018    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:55.581300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.581279    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:55.581373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.581330    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:55.581373 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.581346    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:55.582893 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.582868    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "kube-api-access-8sx2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:41:55.582998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.582966    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:41:55.583436 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.583416    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm" pod "297573e4-27ee-460d-b847-6ed4f8c45c6b" (UID: "297573e4-27ee-460d-b847-6ed4f8c45c6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:41:55.682297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.682210    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sx2t\" (UniqueName: \"kubernetes.io/projected/297573e4-27ee-460d-b847-6ed4f8c45c6b-kube-api-access-8sx2t\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:55.682297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.682247    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297573e4-27ee-460d-b847-6ed4f8c45c6b-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:55.682297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:55.682261    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297573e4-27ee-460d-b847-6ed4f8c45c6b-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:41:56.029017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.028975    4470 generic.go:356] "Generic (PLEG): container finished" podID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerID="111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e" exitCode=0
Nov 02 15:41:56.029017 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.029022    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"
Nov 02 15:41:56.029284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.029032    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" event={"ID":"297573e4-27ee-460d-b847-6ed4f8c45c6b","Type":"ContainerDied","Data":"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"}
Nov 02 15:41:56.029284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.029073    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2" event={"ID":"297573e4-27ee-460d-b847-6ed4f8c45c6b","Type":"ContainerDied","Data":"30bab4bd32c98dd3681e258dd03b8a5e44a35687b8e6896e43613a5c3bc7fcfe"}
Nov 02 15:41:56.029284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.029100    4470 scope.go:122] "RemoveContainer" containerID="111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"
Nov 02 15:41:56.038278 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.038250    4470 scope.go:122] "RemoveContainer" containerID="111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"
Nov 02 15:41:56.038591 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:41:56.038567    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e\": container with ID starting with 111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e not found: ID does not exist" containerID="111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"
Nov 02 15:41:56.038671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.038601    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e"} err="failed to get container status \"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e\": rpc error: code = NotFound desc = could not find container \"111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e\": container with ID starting with 111ba24f240642d6cbf70dd0909af9d2726ff8bd050c410ed4954931f303df0e not found: ID does not exist"
Nov 02 15:41:56.193507 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.193471    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:41:56.201694 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:56.201662    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-019efe/premium-w1-019efe-kserve-d4df5657-96ll2"]
Nov 02 15:41:57.704860 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:41:57.704827    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" path="/var/lib/kubelet/pods/297573e4-27ee-460d-b847-6ed4f8c45c6b/volumes"
Nov 02 15:42:03.565641 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:42:03.565596    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:42:19.381919 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.381832    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:42:19.382469 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.382103    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-76976749fb-92m7t" podUID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" containerName="manager" containerID="cri-o://6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e" gracePeriod=30
Nov 02 15:42:19.647457 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.647389    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:42:19.708740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.708703    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\") pod \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") "
Nov 02 15:42:19.708932 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.708802    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\" (UniqueName: \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\") pod \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") "
Nov 02 15:42:19.708932 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.708839    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\") pod \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\" (UID: \"23e9527e-58d7-4ecc-9cf8-81a0cc60a792\") "
Nov 02 15:42:19.711047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.711016    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf" pod "23e9527e-58d7-4ecc-9cf8-81a0cc60a792" (UID: "23e9527e-58d7-4ecc-9cf8-81a0cc60a792"). InnerVolumeSpecName "kube-api-access-6rmjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:42:19.711047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.711046    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs" pod "23e9527e-58d7-4ecc-9cf8-81a0cc60a792" (UID: "23e9527e-58d7-4ecc-9cf8-81a0cc60a792"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:42:19.711236 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.711138    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert" pod "23e9527e-58d7-4ecc-9cf8-81a0cc60a792" (UID: "23e9527e-58d7-4ecc-9cf8-81a0cc60a792"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:42:19.810876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.810843    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rmjf\" (UniqueName: \"kubernetes.io/projected/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-kube-api-access-6rmjf\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:42:19.810876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.810875    4470 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-webhook-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:42:19.811079 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:19.810885    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23e9527e-58d7-4ecc-9cf8-81a0cc60a792-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:42:20.156942 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.156906    4470 generic.go:356] "Generic (PLEG): container finished" podID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" containerID="6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e" exitCode=0
Nov 02 15:42:20.157141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.156954    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-92m7t"
Nov 02 15:42:20.157141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.156955    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-92m7t" event={"ID":"23e9527e-58d7-4ecc-9cf8-81a0cc60a792","Type":"ContainerDied","Data":"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"}
Nov 02 15:42:20.157141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.157078    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-92m7t" event={"ID":"23e9527e-58d7-4ecc-9cf8-81a0cc60a792","Type":"ContainerDied","Data":"3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560"}
Nov 02 15:42:20.157141 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.157106    4470 scope.go:122] "RemoveContainer" containerID="6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"
Nov 02 15:42:20.168314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.168266    4470 scope.go:122] "RemoveContainer" containerID="6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"
Nov 02 15:42:20.168633 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:42:20.168609    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e\": container with ID starting with 6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e not found: ID does not exist" containerID="6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"
Nov 02 15:42:20.168705 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.168648    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e"} err="failed to get container status \"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e\": rpc error: code = NotFound desc = could not find container \"6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e\": container with ID starting with 6b9046f6bc84940d04779e9e82494bff8065ad9f6d0f143c584e1916ff19389e not found: ID does not exist"
Nov 02 15:42:20.188711 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:42:20.188676    4470 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e9527e_58d7_4ecc_9cf8_81a0cc60a792.slice/crio-3a756fb64dd278c23fc9e25cd7db783202382509dd29aa69ad494d491d00e560\": RecentStats: unable to find data in memory cache]"
Nov 02 15:42:20.322096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.322061    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:42:20.371728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:20.371687    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76976749fb-92m7t"]
Nov 02 15:42:21.704530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:21.704500    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" path="/var/lib/kubelet/pods/23e9527e-58d7-4ecc-9cf8-81a0cc60a792/volumes"
Nov 02 15:42:31.253740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.253707    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:42:31.254140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254129    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" containerName="payload-pre-processing"
Nov 02 15:42:31.254184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254142    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" containerName="payload-pre-processing"
Nov 02 15:42:31.254184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254168    4470 cpu_manager.go:414] "containerMap: removing container" podUID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" containerName="manager"
Nov 02 15:42:31.254184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254173    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" containerName="manager"
Nov 02 15:42:31.254184 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254182    4470 cpu_manager.go:414] "containerMap: removing container" podUID="ce49499e-0667-44c0-813e-8fd7db8f4e33" containerName="revoke-keys"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254202    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce49499e-0667-44c0-813e-8fd7db8f4e33" containerName="revoke-keys"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254215    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" containerName="payload-processing"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254219    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" containerName="payload-processing"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254235    4470 cpu_manager.go:414] "containerMap: removing container" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254240    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254248    4470 cpu_manager.go:414] "containerMap: removing container" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254252    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254314    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4306b86e-471a-4a5d-bdcd-3b679571fcb8" containerName="payload-pre-processing"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254326    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="23e9527e-58d7-4ecc-9cf8-81a0cc60a792" containerName="manager"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254333    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce49499e-0667-44c0-813e-8fd7db8f4e33" containerName="revoke-keys"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254340    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4da8b94-0bcf-4788-b4a8-dde8ebcdede6" containerName="payload-processing"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254348    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="297573e4-27ee-460d-b847-6ed4f8c45c6b" containerName="main"
Nov 02 15:42:31.254362 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.254355    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="745c00bf-701e-461a-8558-28f1eb90e28c" containerName="istio-proxy"
Nov 02 15:42:31.257646 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.257630    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.363007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.362940    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-77ggk\""
Nov 02 15:42:31.363007 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.363005    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 15:42:31.363285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.363028    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 15:42:31.363285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.363169    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 15:42:31.366055 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.366033    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:42:31.369845 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.369813    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k69r\" (UniqueName: \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.369987 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.369876    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.370046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.369991    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.471629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.471600    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k69r\" (UniqueName: \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.471809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.471644    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.471809 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.471667    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.474160 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.474135    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.474291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.474274    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.537166 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.537101    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k69r\" (UniqueName: \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\") pod \"maas-controller-76976749fb-hrhwl\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") " pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:31.570955 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:31.570929    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:32.003300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:32.003000    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:42:32.006780 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:42:32.006751    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8654cf_22f1_4a03_8c03_2c32463e7f46.slice/crio-1771ea6d08471dac9857a6ec0eb88608ac614f83441068e7c55aa3df88aed7a1 WatchSource:0}: Error finding container 1771ea6d08471dac9857a6ec0eb88608ac614f83441068e7c55aa3df88aed7a1: Status 404 returned error can't find the container with id 1771ea6d08471dac9857a6ec0eb88608ac614f83441068e7c55aa3df88aed7a1
Nov 02 15:42:32.008386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:32.008365    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:42:32.271777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:32.271688    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-hrhwl" event={"ID":"be8654cf-22f1-4a03-8c03-2c32463e7f46","Type":"ContainerStarted","Data":"1771ea6d08471dac9857a6ec0eb88608ac614f83441068e7c55aa3df88aed7a1"}
Nov 02 15:42:33.278066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:33.277959    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-hrhwl" event={"ID":"be8654cf-22f1-4a03-8c03-2c32463e7f46","Type":"ContainerStarted","Data":"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"}
Nov 02 15:42:33.278522 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:33.278154    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:42:33.399856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:33.399797    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-76976749fb-hrhwl" podStartSLOduration=1.460098221 podStartE2EDuration="2.399782445s" totalImagesPullingTime="939.684224ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:42:33.397327326 +0000 UTC m=+2759.835229470" watchObservedRunningTime="2026-11-02 15:42:33.399782445 +0000 UTC m=+2759.837684586"
Nov 02 15:42:33.564331 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:42:33.564239    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:42:44.285617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:42:44.285583    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:43:03.564970 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:43:03.564934    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:43:06.159311 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.159271    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:43:06.159828 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.159543    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-76976749fb-hrhwl" podUID="be8654cf-22f1-4a03-8c03-2c32463e7f46" containerName="manager" containerID="cri-o://8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd" gracePeriod=30
Nov 02 15:43:06.416178 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.416112    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:43:06.438244 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.438211    4470 generic.go:356] "Generic (PLEG): container finished" podID="be8654cf-22f1-4a03-8c03-2c32463e7f46" containerID="8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd" exitCode=0
Nov 02 15:43:06.438426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.438294    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-hrhwl"
Nov 02 15:43:06.438426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.438285    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-hrhwl" event={"ID":"be8654cf-22f1-4a03-8c03-2c32463e7f46","Type":"ContainerDied","Data":"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"}
Nov 02 15:43:06.438541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.438443    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-hrhwl" event={"ID":"be8654cf-22f1-4a03-8c03-2c32463e7f46","Type":"ContainerDied","Data":"1771ea6d08471dac9857a6ec0eb88608ac614f83441068e7c55aa3df88aed7a1"}
Nov 02 15:43:06.438541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.438466    4470 scope.go:122] "RemoveContainer" containerID="8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"
Nov 02 15:43:06.447092 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.447072    4470 scope.go:122] "RemoveContainer" containerID="8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"
Nov 02 15:43:06.447421 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:43:06.447397    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd\": container with ID starting with 8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd not found: ID does not exist" containerID="8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"
Nov 02 15:43:06.447510 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.447438    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd"} err="failed to get container status \"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd\": rpc error: code = NotFound desc = could not find container \"8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd\": container with ID starting with 8b7b716a1ab1f12c9f74bc45274eee8615710536e80c4db948811886a0603dcd not found: ID does not exist"
Nov 02 15:43:06.539359 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.539323    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\") pod \"be8654cf-22f1-4a03-8c03-2c32463e7f46\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") "
Nov 02 15:43:06.539515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.539450    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\") pod \"be8654cf-22f1-4a03-8c03-2c32463e7f46\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") "
Nov 02 15:43:06.539515 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.539471    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\" (UniqueName: \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\") pod \"be8654cf-22f1-4a03-8c03-2c32463e7f46\" (UID: \"be8654cf-22f1-4a03-8c03-2c32463e7f46\") "
Nov 02 15:43:06.541888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.541848    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs" pod "be8654cf-22f1-4a03-8c03-2c32463e7f46" (UID: "be8654cf-22f1-4a03-8c03-2c32463e7f46"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:43:06.542643 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.542609    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r" pod "be8654cf-22f1-4a03-8c03-2c32463e7f46" (UID: "be8654cf-22f1-4a03-8c03-2c32463e7f46"). InnerVolumeSpecName "kube-api-access-2k69r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:43:06.542950 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.542905    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert" pod "be8654cf-22f1-4a03-8c03-2c32463e7f46" (UID: "be8654cf-22f1-4a03-8c03-2c32463e7f46"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:43:06.640757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.640710    4470 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-webhook-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:43:06.640757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.640755    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k69r\" (UniqueName: \"kubernetes.io/projected/be8654cf-22f1-4a03-8c03-2c32463e7f46-kube-api-access-2k69r\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:43:06.640757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.640767    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8654cf-22f1-4a03-8c03-2c32463e7f46-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:43:06.862219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.862160    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:43:06.865008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:06.864981    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-76976749fb-hrhwl"]
Nov 02 15:43:07.705122 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:07.705093    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be8654cf-22f1-4a03-8c03-2c32463e7f46" path="/var/lib/kubelet/pods/be8654cf-22f1-4a03-8c03-2c32463e7f46/volumes"
Nov 02 15:43:08.354337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.354298    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-76976749fb-gn8d9"]
Nov 02 15:43:08.354764 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.354751    4470 cpu_manager.go:414] "containerMap: removing container" podUID="be8654cf-22f1-4a03-8c03-2c32463e7f46" containerName="manager"
Nov 02 15:43:08.354820 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.354765    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be8654cf-22f1-4a03-8c03-2c32463e7f46" containerName="manager"
Nov 02 15:43:08.354856 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.354831    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="be8654cf-22f1-4a03-8c03-2c32463e7f46" containerName="manager"
Nov 02 15:43:08.359338 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.359321    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.460798 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.460758    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-metrics-certs\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.461072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.460850    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94mkl\" (UniqueName: \"kubernetes.io/projected/5ef0279a-f41b-46ac-98e4-aa329ff8395f-kube-api-access-94mkl\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.461072 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.460949    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-webhook-cert\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.461237 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.461214    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 02 15:43:08.461564 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.461544    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-77ggk\""
Nov 02 15:43:08.461991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.461972    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 02 15:43:08.461991 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.461975    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 02 15:43:08.464239 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.464219    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-gn8d9"]
Nov 02 15:43:08.561911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.561875    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-webhook-cert\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.562118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.561945    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-metrics-certs\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.562118 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.562017    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94mkl\" (UniqueName: \"kubernetes.io/projected/5ef0279a-f41b-46ac-98e4-aa329ff8395f-kube-api-access-94mkl\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.564449 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.564424    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-metrics-certs\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.564579 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.564468    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5ef0279a-f41b-46ac-98e4-aa329ff8395f-webhook-cert\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.623404 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.623324    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94mkl\" (UniqueName: \"kubernetes.io/projected/5ef0279a-f41b-46ac-98e4-aa329ff8395f-kube-api-access-94mkl\") pod \"maas-controller-76976749fb-gn8d9\" (UID: \"5ef0279a-f41b-46ac-98e4-aa329ff8395f\") " pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:08.672667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:08.672623    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:09.056825 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:09.056796    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-76976749fb-gn8d9"]
Nov 02 15:43:09.058576 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:43:09.058547    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef0279a_f41b_46ac_98e4_aa329ff8395f.slice/crio-8fcc2c7da4a299b79ae1429ddb4e215cf007be90441c4780e8b7947b04ff8bf4 WatchSource:0}: Error finding container 8fcc2c7da4a299b79ae1429ddb4e215cf007be90441c4780e8b7947b04ff8bf4: Status 404 returned error can't find the container with id 8fcc2c7da4a299b79ae1429ddb4e215cf007be90441c4780e8b7947b04ff8bf4
Nov 02 15:43:09.455568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:09.455529    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-gn8d9" event={"ID":"5ef0279a-f41b-46ac-98e4-aa329ff8395f","Type":"ContainerStarted","Data":"8fcc2c7da4a299b79ae1429ddb4e215cf007be90441c4780e8b7947b04ff8bf4"}
Nov 02 15:43:10.461979 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:10.461941    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-76976749fb-gn8d9" event={"ID":"5ef0279a-f41b-46ac-98e4-aa329ff8395f","Type":"ContainerStarted","Data":"827ee099d5ccbdcfebdc86f8d72160644dd25c7342bfc45815d881618eb9ff46"}
Nov 02 15:43:10.462485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:10.462105    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:10.585014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:10.584957    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-76976749fb-gn8d9" podStartSLOduration=1.793563314 podStartE2EDuration="2.584943311s" totalImagesPullingTime="791.379997ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:43:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:43:10.583068353 +0000 UTC m=+2797.020970506" watchObservedRunningTime="2026-11-02 15:43:10.584943311 +0000 UTC m=+2797.022845488"
Nov 02 15:43:21.470707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:43:21.470675    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-76976749fb-gn8d9"
Nov 02 15:43:33.564616 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:43:33.564572    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:44:03.565853 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:44:03.565805    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:44:33.565135 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:44:33.565095    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:45:03.564736 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:45:03.564694    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:45:33.564261 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:45:33.564224    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:46:03.564996 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:46:03.564953    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:46:33.565807 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:46:33.565764    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:46:34.111525 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:46:34.111495    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:46:34.155562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:46:34.155539    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:46:53.065593 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:46:53.065506    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"]
Nov 02 15:46:53.114505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:46:53.114468    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76eaf9b0-2f26p"]
Nov 02 15:46:53.704948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:46:53.704915    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce49499e-0667-44c0-813e-8fd7db8f4e33" path="/var/lib/kubelet/pods/ce49499e-0667-44c0-813e-8fd7db8f4e33/volumes"
Nov 02 15:47:03.565617 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:47:03.565569    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:47:33.565151 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:47:33.565106    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:48:03.563704 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:48:03.563665    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:48:27.802535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:48:27.802439    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" podUID="6e7f87aa5e9c9e37d740f6ee1d873224" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Nov 02 15:48:33.565649 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:48:33.565605    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:48:35.207812 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:48:35.207775    4470 scope.go:122] "RemoveContainer" containerID="c5bf51dd5d74b00bf0538cd48e4dce87f58dcb733996d273fd42be380f962b00"
Nov 02 15:49:03.564583 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:49:03.564535    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:49:33.564556 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:49:33.564514    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:50:03.565595 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:50:03.565472    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:50:33.566535 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:50:33.566487    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:50:39.935284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:39.935245    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:50:39.942148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:39.942111    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:39.994799 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:39.994755    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-b72a1f-openshift-default-dockercfg-ggshj\""
Nov 02 15:50:40.004258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004215    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004283    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004371    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004468    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004617 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004515    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpln9\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004626    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004665    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004694    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.004897 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.004762    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.046667 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.046628    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:50:40.105654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105617    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105658    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpln9\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105703    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105730    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105751    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105766    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105792    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.105928 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105873    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.105952    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.106159    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.106405    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.106434    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106511 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.106475    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.106736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.106715    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.108583 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.108559    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.108785 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.108767    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.167177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.167135    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.167403 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.167277    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpln9\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\") pod \"e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") " pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.265133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.265047    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:40.695474 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.695404    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:50:40.697957 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:50:40.697929    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d35b49b_b2ce_4310_9533_1bf1da0cd491.slice/crio-cdd574416769e0747743ecb29ac1e3eac56eb9534dccb59fb63857348b6600a2 WatchSource:0}: Error finding container cdd574416769e0747743ecb29ac1e3eac56eb9534dccb59fb63857348b6600a2: Status 404 returned error can't find the container with id cdd574416769e0747743ecb29ac1e3eac56eb9534dccb59fb63857348b6600a2
Nov 02 15:50:40.700298 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.700264    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:50:40.700370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.700335    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:50:40.700370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.700364    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:50:40.877994 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.877959    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerStarted","Data":"20a343e501ff0b972f4e57d95053c62c0f7fae788a0c1b7bb5ecfad7b0143917"}
Nov 02 15:50:40.878176 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:40.878012    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerStarted","Data":"cdd574416769e0747743ecb29ac1e3eac56eb9534dccb59fb63857348b6600a2"}
Nov 02 15:50:41.000568 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:41.000448    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podStartSLOduration=2.000427614 podStartE2EDuration="2.000427614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:50:40.999355508 +0000 UTC m=+3247.437257648" watchObservedRunningTime="2026-11-02 15:50:41.000427614 +0000 UTC m=+3247.438329756"
Nov 02 15:50:41.265654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:41.265548    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:41.270732 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:41.270703    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:41.883446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:41.883413    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:41.884470 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:41.884449    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:50:47.803791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.803751    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:50:47.807806 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.807783    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:47.859993 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.859966    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-b72a1f\""
Nov 02 15:50:47.883692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.883648    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:47.883913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.883881    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f6h7\" (UniqueName: \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:47.925445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.925400    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:50:47.948048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.948009    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:50:47.952636 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.952560    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:47.985518 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.985484    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f6h7\" (UniqueName: \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:47.985706 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.985554    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:47.986277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:47.986254    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:48.005284 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.005250    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-e886357c-dockercfg-qlvzr\""
Nov 02 15:50:48.006546 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.006519    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:50:48.046535 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.046497    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f6h7\" (UniqueName: \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\") pod \"payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:48.086327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.086244    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qljzm\" (UniqueName: \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.086327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.086308    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.121091 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.121061    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:48.188087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.188042    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qljzm\" (UniqueName: \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.188576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.188531    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.189391 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.189367    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.249662 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.249629    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qljzm\" (UniqueName: \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\") pod \"payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.268299 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.268267    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.502736 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.502702    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:50:48.505294 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:50:48.505263    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5910e121_203f_430d_96ba_bc095125009b.slice/crio-ea2a62cd940f123910dc91e26f52bc39b41aa9cd55168284d841450c73b0e3c0 WatchSource:0}: Error finding container ea2a62cd940f123910dc91e26f52bc39b41aa9cd55168284d841450c73b0e3c0: Status 404 returned error can't find the container with id ea2a62cd940f123910dc91e26f52bc39b41aa9cd55168284d841450c73b0e3c0
Nov 02 15:50:48.643891 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.643800    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:50:48.646274 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:50:48.646239    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f70ddc0_e19c_4fb5_9a67_bd8e3a2e4b36.slice/crio-f51c6672b0d416e6ec1e7ee0c229a85ed76b13ce2f5955eef41a3cc95991c2c3 WatchSource:0}: Error finding container f51c6672b0d416e6ec1e7ee0c229a85ed76b13ce2f5955eef41a3cc95991c2c3: Status 404 returned error can't find the container with id f51c6672b0d416e6ec1e7ee0c229a85ed76b13ce2f5955eef41a3cc95991c2c3
Nov 02 15:50:48.936687 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.936653    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" event={"ID":"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36","Type":"ContainerStarted","Data":"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"}
Nov 02 15:50:48.937208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.936705    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" event={"ID":"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36","Type":"ContainerStarted","Data":"f51c6672b0d416e6ec1e7ee0c229a85ed76b13ce2f5955eef41a3cc95991c2c3"}
Nov 02 15:50:48.937208 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.936761    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:50:48.938443 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.938418    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" event={"ID":"5910e121-203f-430d-96ba-bc095125009b","Type":"ContainerStarted","Data":"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"}
Nov 02 15:50:48.938563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.938462    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" event={"ID":"5910e121-203f-430d-96ba-bc095125009b","Type":"ContainerStarted","Data":"ea2a62cd940f123910dc91e26f52bc39b41aa9cd55168284d841450c73b0e3c0"}
Nov 02 15:50:48.938563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:48.938524    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:49.054291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:49.054234    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" podStartSLOduration=2.054216983 podStartE2EDuration="2.054216983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:50:49.052479466 +0000 UTC m=+3255.490381616" watchObservedRunningTime="2026-11-02 15:50:49.054216983 +0000 UTC m=+3255.492119124"
Nov 02 15:50:50.144873 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.144817    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" podStartSLOduration=3.144801131 podStartE2EDuration="3.144801131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:50:49.175836276 +0000 UTC m=+3255.613738545" watchObservedRunningTime="2026-11-02 15:50:50.144801131 +0000 UTC m=+3256.582703271"
Nov 02 15:50:50.146475 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.146454    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:50:50.150301 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.150278    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.202969 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.202938    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-b72a1f-serving-cert\""
Nov 02 15:50:50.211998 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.211967    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:50:50.309324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.309282    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.309493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.309329    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.309493 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.309436    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdxc9\" (UniqueName: \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.309576 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.309551    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.411140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.411039    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.411140 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.411098    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.411399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.411160    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdxc9\" (UniqueName: \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.411399 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.411288    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.411995 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.411967    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.419777 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.419742    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.420773 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.420741    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.472965 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.472927    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdxc9\" (UniqueName: \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\") pod \"maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:50.764386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:50.764343    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:51.147050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:51.147021    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:50:51.149078 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:50:51.149049    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9691825_a971_4391_b102_31db0d583579.slice/crio-185d42c698f845c95f1a25daec9cda4c3423380f465e3176b3d994d18b9dc6a2 WatchSource:0}: Error finding container 185d42c698f845c95f1a25daec9cda4c3423380f465e3176b3d994d18b9dc6a2: Status 404 returned error can't find the container with id 185d42c698f845c95f1a25daec9cda4c3423380f465e3176b3d994d18b9dc6a2
Nov 02 15:50:51.150578 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:51.150560    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 15:50:51.956382 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:51.956344    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" event={"ID":"e9691825-a971-4391-b102-31db0d583579","Type":"ContainerStarted","Data":"185d42c698f845c95f1a25daec9cda4c3423380f465e3176b3d994d18b9dc6a2"}
Nov 02 15:50:52.962505 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:52.962468    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" event={"ID":"e9691825-a971-4391-b102-31db0d583579","Type":"ContainerStarted","Data":"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"}
Nov 02 15:50:52.963018 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:52.962669    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:53.081177 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:53.081118    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" podStartSLOduration=2.273688849 podStartE2EDuration="3.081103537s" totalImagesPullingTime="807.414688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:50:53.079859133 +0000 UTC m=+3259.517761280" watchObservedRunningTime="2026-11-02 15:50:53.081103537 +0000 UTC m=+3259.519005677"
Nov 02 15:50:58.995215 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:58.994784    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:50:59.946082 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:59.946051    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:50:59.946275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:50:59.946120    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:51:03.564735 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:51:03.564695    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:51:08.046634 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.046584    4470 generic.go:356] "Generic (PLEG): container finished" podID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerID="20a343e501ff0b972f4e57d95053c62c0f7fae788a0c1b7bb5ecfad7b0143917" exitCode=0
Nov 02 15:51:08.047102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.046657    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerDied","Data":"20a343e501ff0b972f4e57d95053c62c0f7fae788a0c1b7bb5ecfad7b0143917"}
Nov 02 15:51:08.047102 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.047066    4470 scope.go:122] "RemoveContainer" containerID="20a343e501ff0b972f4e57d95053c62c0f7fae788a0c1b7bb5ecfad7b0143917"
Nov 02 15:51:08.048485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.048458    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:51:08.048562 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.048528    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:51:08.048610 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:08.048561    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:51:09.053776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:09.053739    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerStarted","Data":"2fc36382ab007b646c21c1eb67f2f9b0a468f2f258ca35c4244cd96318c3a47b"}
Nov 02 15:51:09.266204 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:09.266162    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:51:09.267367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:09.267328    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Nov 02 15:51:09.267529 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:09.267401    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Nov 02 15:51:10.266155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:10.266109    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Nov 02 15:51:10.266615 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:10.266218    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Nov 02 15:51:11.267039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:11.266419    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Nov 02 15:51:11.267039 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:11.266499    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Nov 02 15:51:11.891216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:11.890315    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:51:12.265494 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:12.265457    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Nov 02 15:51:12.265684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:12.265513    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Nov 02 15:51:13.266509 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.266467    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused" start-of-body=
Nov 02 15:51:13.266935 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.266529    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.146:15021/healthz/ready\": dial tcp 10.133.0.146:15021: connect: connection refused"
Nov 02 15:51:13.718328 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.718293    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:51:13.721229 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.721207    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.764943 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.764899    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.765180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.764958    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.765180 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.765121    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.765348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.765242    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.765348 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.765279    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.765424 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.765383    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4spzj\" (UniqueName: \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.796808 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.796772    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-b72a1f\"/\"openshift-service-ca.crt\""
Nov 02 15:51:13.797337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.797258    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-b72a1f\"/\"kube-root-ca.crt\""
Nov 02 15:51:13.829603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.829572    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-b72a1f\"/\"default-dockercfg-gqxbh\""
Nov 02 15:51:13.829816 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.829658    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-b72a1f\"/\"simulator-single-main-b72a1f-kserve-self-signed-certs\""
Nov 02 15:51:13.831960 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.831935    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:51:13.866603 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866565    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.866807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866624    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.866807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866713    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.866807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866770    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.866807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866788    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.866970 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.866827    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4spzj\" (UniqueName: \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.867778 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.867717    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.867778 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.867738    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.868014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.867993    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.896933 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.896889    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.904525 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.904493    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:13.928978 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:13.928936    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4spzj\" (UniqueName: \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\") pod \"simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") " pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:14.032495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:14.032400    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:14.270461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:14.270417    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:51:14.271604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:14.271582    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:51:14.411541 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:14.411481    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:51:14.413831 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:51:14.413801    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc23722cc_ef03_4355_8aac_20373498ca26.slice/crio-4bcb49c81aaa635886f11268c3b9cc2f345e860d94ec97c9f2d668e82828fa24 WatchSource:0}: Error finding container 4bcb49c81aaa635886f11268c3b9cc2f345e860d94ec97c9f2d668e82828fa24: Status 404 returned error can't find the container with id 4bcb49c81aaa635886f11268c3b9cc2f345e860d94ec97c9f2d668e82828fa24
Nov 02 15:51:15.112698 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:15.112659    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" event={"ID":"c23722cc-ef03-4355-8aac-20373498ca26","Type":"ContainerStarted","Data":"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"}
Nov 02 15:51:15.112903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:15.112712    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" event={"ID":"c23722cc-ef03-4355-8aac-20373498ca26","Type":"ContainerStarted","Data":"4bcb49c81aaa635886f11268c3b9cc2f345e860d94ec97c9f2d668e82828fa24"}
Nov 02 15:51:15.234453 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:15.234381    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" podStartSLOduration=2.234361036 podStartE2EDuration="2.234361036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:51:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:51:15.23251327 +0000 UTC m=+3281.670415413" watchObservedRunningTime="2026-11-02 15:51:15.234361036 +0000 UTC m=+3281.672263180"
Nov 02 15:51:16.033386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:16.033352    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:24.032600 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:24.032561    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:24.045406 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:24.045374    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:24.169219 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:24.169172    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:51:33.564738 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:51:33.564694    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:51:34.144506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:34.144477    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:51:34.191832 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:51:34.191807    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:52:03.566057 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:03.566007    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:52:26.795324 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:26.795286    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:52:33.565389 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:33.565350    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:52:33.898315 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:33.898227    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:52:34.004671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:34.004629    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:52:36.913170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:36.913131    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:52:36.913742 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:36.913348    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" podUID="e9691825-a971-4391-b102-31db0d583579" containerName="maas-api" containerID="cri-o://0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a" gracePeriod=30
Nov 02 15:52:38.175096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.175063    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:52:38.296031 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.295946    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\") pod \"e9691825-a971-4391-b102-31db0d583579\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") "
Nov 02 15:52:38.296213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.296140    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\") pod \"e9691825-a971-4391-b102-31db0d583579\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") "
Nov 02 15:52:38.296213 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.296170    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\") pod \"e9691825-a971-4391-b102-31db0d583579\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") "
Nov 02 15:52:38.296334 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.296211    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\" (UniqueName: \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\") pod \"e9691825-a971-4391-b102-31db0d583579\" (UID: \"e9691825-a971-4391-b102-31db0d583579\") "
Nov 02 15:52:38.296569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.296541    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca" pod "e9691825-a971-4391-b102-31db0d583579" (UID: "e9691825-a971-4391-b102-31db0d583579"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:52:38.298789 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.298760    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs" pod "e9691825-a971-4391-b102-31db0d583579" (UID: "e9691825-a971-4391-b102-31db0d583579"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:52:38.298910 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.298762    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9" pod "e9691825-a971-4391-b102-31db0d583579" (UID: "e9691825-a971-4391-b102-31db0d583579"). InnerVolumeSpecName "kube-api-access-qdxc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:38.299295 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.299274    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls" pod "e9691825-a971-4391-b102-31db0d583579" (UID: "e9691825-a971-4391-b102-31db0d583579"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:52:38.397506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.397471    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:38.397506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.397502    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9691825-a971-4391-b102-31db0d583579-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:38.397506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.397515    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9691825-a971-4391-b102-31db0d583579-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:38.397765 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.397524    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdxc9\" (UniqueName: \"kubernetes.io/projected/e9691825-a971-4391-b102-31db0d583579-kube-api-access-qdxc9\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:38.544435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.544394    4470 generic.go:356] "Generic (PLEG): container finished" podID="e9691825-a971-4391-b102-31db0d583579" containerID="0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a" exitCode=0
Nov 02 15:52:38.544626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.544445    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"
Nov 02 15:52:38.544626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.544468    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" event={"ID":"e9691825-a971-4391-b102-31db0d583579","Type":"ContainerDied","Data":"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"}
Nov 02 15:52:38.544626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.544527    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt" event={"ID":"e9691825-a971-4391-b102-31db0d583579","Type":"ContainerDied","Data":"185d42c698f845c95f1a25daec9cda4c3423380f465e3176b3d994d18b9dc6a2"}
Nov 02 15:52:38.544626 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.544552    4470 scope.go:122] "RemoveContainer" containerID="0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"
Nov 02 15:52:38.556904 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.556884    4470 scope.go:122] "RemoveContainer" containerID="0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"
Nov 02 15:52:38.557246 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:38.557224    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a\": container with ID starting with 0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a not found: ID does not exist" containerID="0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"
Nov 02 15:52:38.557303 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.557257    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a"} err="failed to get container status \"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a\": rpc error: code = NotFound desc = could not find container \"0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a\": container with ID starting with 0e52cfcf32e9915df53ffe02fd6e3ea2935cc28928820e22f8b627ab7b558b8a not found: ID does not exist"
Nov 02 15:52:38.716506 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.716468    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:52:38.765954 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.765919    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b72a1f-8689847579-dq6vt"]
Nov 02 15:52:38.898846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:38.898735    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing" containerID="cri-o://3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a" gracePeriod=25
Nov 02 15:52:39.005326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:39.005251    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing" containerID="cri-o://4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4" gracePeriod=25
Nov 02 15:52:39.705781 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:39.705744    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9691825-a971-4391-b102-31db0d583579" path="/var/lib/kubelet/pods/e9691825-a971-4391-b102-31db0d583579/volumes"
Nov 02 15:52:39.945473 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:39.945429    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:52:39.945653 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:39.945441    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:52:41.816554 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:41.816507    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" podUID="c23722cc-ef03-4355-8aac-20373498ca26" containerName="main" containerID="cri-o://7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907" gracePeriod=45
Nov 02 15:52:42.058355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.058326    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:52:42.137866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.137765    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.137866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.137839    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.138132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.137866    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.138132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.137917    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\" (UniqueName: \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.138132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.137958    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.138132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138044    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\" (UniqueName: \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\") pod \"c23722cc-ef03-4355-8aac-20373498ca26\" (UID: \"c23722cc-ef03-4355-8aac-20373498ca26\") "
Nov 02 15:52:42.138132 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138118    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:42.138447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138288    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:42.138447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138407    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.138447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138417    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:42.138620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.138427    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.140510 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.140479    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:52:42.140630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.140533    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:42.140630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.140537    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj" pod "c23722cc-ef03-4355-8aac-20373498ca26" (UID: "c23722cc-ef03-4355-8aac-20373498ca26"). InnerVolumeSpecName "kube-api-access-4spzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:42.239539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.239502    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.239539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.239534    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c23722cc-ef03-4355-8aac-20373498ca26-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.239539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.239546    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c23722cc-ef03-4355-8aac-20373498ca26-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.239784 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.239559    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4spzj\" (UniqueName: \"kubernetes.io/projected/c23722cc-ef03-4355-8aac-20373498ca26-kube-api-access-4spzj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:42.568133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.568094    4470 generic.go:356] "Generic (PLEG): container finished" podID="c23722cc-ef03-4355-8aac-20373498ca26" containerID="7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907" exitCode=0
Nov 02 15:52:42.568133 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.568143    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"
Nov 02 15:52:42.568463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.568181    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" event={"ID":"c23722cc-ef03-4355-8aac-20373498ca26","Type":"ContainerDied","Data":"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"}
Nov 02 15:52:42.568463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.568252    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m" event={"ID":"c23722cc-ef03-4355-8aac-20373498ca26","Type":"ContainerDied","Data":"4bcb49c81aaa635886f11268c3b9cc2f345e860d94ec97c9f2d668e82828fa24"}
Nov 02 15:52:42.568463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.568271    4470 scope.go:122] "RemoveContainer" containerID="7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"
Nov 02 15:52:42.578054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.578038    4470 scope.go:122] "RemoveContainer" containerID="7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"
Nov 02 15:52:42.578374 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:42.578356    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907\": container with ID starting with 7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907 not found: ID does not exist" containerID="7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"
Nov 02 15:52:42.578422 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.578384    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907"} err="failed to get container status \"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907\": rpc error: code = NotFound desc = could not find container \"7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907\": container with ID starting with 7f9980a3d6049384a951717d20704f061539daed500c369a4016625978c03907 not found: ID does not exist"
Nov 02 15:52:42.736274 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.736239    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:52:42.738123 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:42.738099    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-b72a1f/simulator-single-main-b72a1f-kserve-7f6b67c859-mh64m"]
Nov 02 15:52:43.706728 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:43.706692    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c23722cc-ef03-4355-8aac-20373498ca26" path="/var/lib/kubelet/pods/c23722cc-ef03-4355-8aac-20373498ca26/volumes"
Nov 02 15:52:46.400761 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:46.400672    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:52:46.401231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:46.400887    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy" containerID="cri-o://2fc36382ab007b646c21c1eb67f2f9b0a468f2f258ca35c4244cd96318c3a47b" gracePeriod=30
Nov 02 15:52:49.944553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:49.944506    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:52:49.944553 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:49.944541    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:52:51.618782 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.618745    4470 generic.go:356] "Generic (PLEG): container finished" podID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerID="2fc36382ab007b646c21c1eb67f2f9b0a468f2f258ca35c4244cd96318c3a47b" exitCode=0
Nov 02 15:52:51.619277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.618819    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerDied","Data":"2fc36382ab007b646c21c1eb67f2f9b0a468f2f258ca35c4244cd96318c3a47b"}
Nov 02 15:52:51.619277 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.618887    4470 scope.go:122] "RemoveContainer" containerID="20a343e501ff0b972f4e57d95053c62c0f7fae788a0c1b7bb5ecfad7b0143917"
Nov 02 15:52:51.656787 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.656760    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:52:51.745613 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745569    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.745831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745653    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.745831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745683    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.745831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745704    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.745831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745732    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.745831 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745781    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745860    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745920    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.745950    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\") pod \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\" (UID: \"3d35b49b-b2ce-4310-9533-1bf1da0cd491\") "
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746037    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746060    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:51.746119 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746036    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:51.746495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746328    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:52:51.746495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746465    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.746495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746487    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.746668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746503    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.746668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746518    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.746668 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.746624    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:51.748062 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.748035    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:52:51.748175 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.748074    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:51.748370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.748351    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:52:51.748411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.748365    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9" pod "3d35b49b-b2ce-4310-9533-1bf1da0cd491" (UID: "3d35b49b-b2ce-4310-9533-1bf1da0cd491"). InnerVolumeSpecName "kube-api-access-cpln9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:51.847167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.847133    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.847167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.847162    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.847167 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.847173    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.847447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.847182    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d35b49b-b2ce-4310-9533-1bf1da0cd491-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:51.847447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:51.847223    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpln9\" (UniqueName: \"kubernetes.io/projected/3d35b49b-b2ce-4310-9533-1bf1da0cd491-kube-api-access-cpln9\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:52.625423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:52.625384    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"
Nov 02 15:52:52.625423 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:52.625396    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl" event={"ID":"3d35b49b-b2ce-4310-9533-1bf1da0cd491","Type":"ContainerDied","Data":"cdd574416769e0747743ecb29ac1e3eac56eb9534dccb59fb63857348b6600a2"}
Nov 02 15:52:52.625953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:52.625453    4470 scope.go:122] "RemoveContainer" containerID="2fc36382ab007b646c21c1eb67f2f9b0a468f2f258ca35c4244cd96318c3a47b"
Nov 02 15:52:52.838896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:52.838860    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:52:52.888656 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:52.888573    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b72a1f-openshift-default-75f6bf88c7hdsbl"]
Nov 02 15:52:53.704631 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:53.704591    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" path="/var/lib/kubelet/pods/3d35b49b-b2ce-4310-9533-1bf1da0cd491/volumes"
Nov 02 15:52:59.168231 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.168134    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:52:59.259009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.258984    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:52:59.329262 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.329226    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\" (UniqueName: \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\") pod \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") "
Nov 02 15:52:59.329435 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.329312    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\") pod \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\" (UID: \"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36\") "
Nov 02 15:52:59.329692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.329667    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume" pod "6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" (UID: "6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:52:59.331504 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.331477    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm" pod "6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" (UID: "6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36"). InnerVolumeSpecName "kube-api-access-qljzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:59.430230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430125    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\") pod \"5910e121-203f-430d-96ba-bc095125009b\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") "
Nov 02 15:52:59.430412 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430238    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\" (UniqueName: \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\") pod \"5910e121-203f-430d-96ba-bc095125009b\" (UID: \"5910e121-203f-430d-96ba-bc095125009b\") "
Nov 02 15:52:59.430520 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430495    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume" pod "5910e121-203f-430d-96ba-bc095125009b" (UID: "5910e121-203f-430d-96ba-bc095125009b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:52:59.430694 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430677    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qljzm\" (UniqueName: \"kubernetes.io/projected/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-kube-api-access-qljzm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:59.430694 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430693    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:59.430839 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.430705    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5910e121-203f-430d-96ba-bc095125009b-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:59.432356 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.432329    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7" pod "5910e121-203f-430d-96ba-bc095125009b" (UID: "5910e121-203f-430d-96ba-bc095125009b"). InnerVolumeSpecName "kube-api-access-7f6h7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:52:59.532107 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.532057    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f6h7\" (UniqueName: \"kubernetes.io/projected/5910e121-203f-430d-96ba-bc095125009b-kube-api-access-7f6h7\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:52:59.660822 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.660788    4470 generic.go:356] "Generic (PLEG): container finished" podID="5910e121-203f-430d-96ba-bc095125009b" containerID="4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4" exitCode=0
Nov 02 15:52:59.660977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.660840    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"
Nov 02 15:52:59.660977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.660885    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" event={"ID":"5910e121-203f-430d-96ba-bc095125009b","Type":"ContainerDied","Data":"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"}
Nov 02 15:52:59.660977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.660940    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz" event={"ID":"5910e121-203f-430d-96ba-bc095125009b","Type":"ContainerDied","Data":"ea2a62cd940f123910dc91e26f52bc39b41aa9cd55168284d841450c73b0e3c0"}
Nov 02 15:52:59.660977 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.660965    4470 scope.go:122] "RemoveContainer" containerID="4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"
Nov 02 15:52:59.662355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.662334    4470 generic.go:356] "Generic (PLEG): container finished" podID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerID="3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a" exitCode=0
Nov 02 15:52:59.662476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.662370    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" event={"ID":"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36","Type":"ContainerDied","Data":"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"}
Nov 02 15:52:59.662476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.662378    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"
Nov 02 15:52:59.662476 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.662409    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv" event={"ID":"6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36","Type":"ContainerDied","Data":"f51c6672b0d416e6ec1e7ee0c229a85ed76b13ce2f5955eef41a3cc95991c2c3"}
Nov 02 15:52:59.680807 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.680777    4470 scope.go:122] "RemoveContainer" containerID="4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"
Nov 02 15:52:59.681626 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:59.681599    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4\": container with ID starting with 4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4 not found: ID does not exist" containerID="4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"
Nov 02 15:52:59.681753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.681631    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4"} err="failed to get container status \"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4\": rpc error: code = NotFound desc = could not find container \"4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4\": container with ID starting with 4ed72595a2884158e627d36a48ef4173cf75fd15856b462323ffdbde2f8c7ab4 not found: ID does not exist"
Nov 02 15:52:59.681753 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.681652    4470 scope.go:122] "RemoveContainer" containerID="3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"
Nov 02 15:52:59.689909 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.689887    4470 scope.go:122] "RemoveContainer" containerID="3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"
Nov 02 15:52:59.690215 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:52:59.690173    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a\": container with ID starting with 3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a not found: ID does not exist" containerID="3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"
Nov 02 15:52:59.690312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.690222    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a"} err="failed to get container status \"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a\": rpc error: code = NotFound desc = could not find container \"3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a\": container with ID starting with 3a5400da7418213c59d7cd2fec6ff45a1afcbdace43e60ca50759e53d2d9020a not found: ID does not exist"
Nov 02 15:52:59.824282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.824244    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:52:59.828327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.828297    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-b72a1f-5449bcvwrz"]
Nov 02 15:52:59.937486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.937459    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:52:59.941710 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:52:59.941684    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-b72a1f-56b68d794wqpwv"]
Nov 02 15:53:01.704905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:53:01.704868    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5910e121-203f-430d-96ba-bc095125009b" path="/var/lib/kubelet/pods/5910e121-203f-430d-96ba-bc095125009b/volumes"
Nov 02 15:53:01.705337 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:53:01.705269    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" path="/var/lib/kubelet/pods/6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36/volumes"
Nov 02 15:53:03.690214 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:53:03.689547    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:53:33.565440 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:53:33.565393    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:54:03.564490 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:54:03.564449    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:54:33.566011 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:54:33.565925    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:54:49.185740 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:54:49.185672    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:54:49.186772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:54:49.186715    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-cw6h6" podUID="0169e1af-5286-420d-8fe8-96f88517b525" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:55:03.565222 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:55:03.565163    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:55:12.085380 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:12.085329    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:55:12.086260 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:12.086218    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 15:55:33.565346 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:55:33.565305    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:55:33.651881 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.651845    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:55:33.652405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652383    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c23722cc-ef03-4355-8aac-20373498ca26" containerName="main"
Nov 02 15:55:33.652405 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652406    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23722cc-ef03-4355-8aac-20373498ca26" containerName="main"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652426    4470 cpu_manager.go:414] "containerMap: removing container" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652434    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652462    4470 cpu_manager.go:414] "containerMap: removing container" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652470    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652486    4470 cpu_manager.go:414] "containerMap: removing container" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652493    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652525    4470 cpu_manager.go:414] "containerMap: removing container" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652534    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652564    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e9691825-a971-4391-b102-31db0d583579" containerName="maas-api"
Nov 02 15:55:33.652588 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652574    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9691825-a971-4391-b102-31db0d583579" containerName="maas-api"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652684    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23722cc-ef03-4355-8aac-20373498ca26" containerName="main"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652699    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="5910e121-203f-430d-96ba-bc095125009b" containerName="payload-pre-processing"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652713    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652723    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d35b49b-b2ce-4310-9533-1bf1da0cd491" containerName="istio-proxy"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652731    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f70ddc0-e19c-4fb5-9a67-bd8e3a2e4b36" containerName="payload-processing"
Nov 02 15:55:33.653099 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.652744    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9691825-a971-4391-b102-31db0d583579" containerName="maas-api"
Nov 02 15:55:33.656272 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.656249    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.709428 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.709400    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-a60cf3-openshift-default-dockercfg-ptdkq\""
Nov 02 15:55:33.739227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739168    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739272    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739432 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739383    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739443    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739530 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739494    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739597 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739548    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739599    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739729 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739707    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvtpb\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.739783 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.739767    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.761367 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.761334    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:55:33.844054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.843961    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844018    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844054 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844051    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844077    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvtpb\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844105    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844166    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844274    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844328    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844377    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844458    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844502    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844715    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844750    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.844877 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.844845    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.846819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.846795    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.847038 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.847017    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.917925 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.917885    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.918125 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.918027    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvtpb\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\") pod \"e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") " pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:33.969802 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:33.969767    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:34.406162 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.406131    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:55:34.409071 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:55:34.409041    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308d1dc3_c38c_4d5c_933f_7f2789096f51.slice/crio-e814e7bc186ec417f3b3d9116d0e5f44c852511481f577f8de42ae26fee10014 WatchSource:0}: Error finding container e814e7bc186ec417f3b3d9116d0e5f44c852511481f577f8de42ae26fee10014: Status 404 returned error can't find the container with id e814e7bc186ec417f3b3d9116d0e5f44c852511481f577f8de42ae26fee10014
Nov 02 15:55:34.411266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.411230    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:55:34.411353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.411303    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:55:34.411353 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.411336    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:55:34.576833 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.576788    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerStarted","Data":"13789cdc4a6bb714cd02e3bfc134919f998305a2f836648466e22a3719ad011f"}
Nov 02 15:55:34.577320 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.576860    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerStarted","Data":"e814e7bc186ec417f3b3d9116d0e5f44c852511481f577f8de42ae26fee10014"}
Nov 02 15:55:34.702245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.702158    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podStartSLOduration=1.702138486 podStartE2EDuration="1.702138486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:55:34.699364745 +0000 UTC m=+3541.137266887" watchObservedRunningTime="2026-11-02 15:55:34.702138486 +0000 UTC m=+3541.140040627"
Nov 02 15:55:34.970942 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:34.970833    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:35.975848 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:35.975817    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:36.586495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:36.586467    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:36.587862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:36.587835    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:55:41.508504 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.508464    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:55:41.512447 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.512421    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.614427 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.614397    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-a60cf3\""
Nov 02 15:55:41.614620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.614442    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-410515dc-dockercfg-dh85l\""
Nov 02 15:55:41.616630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.616605    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:55:41.631711 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.631676    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.631876 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.631833    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86lsm\" (UniqueName: \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.668569 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.668535    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:55:41.672241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.672222    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.730606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.730570    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:55:41.732866 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.732836    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86lsm\" (UniqueName: \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.733047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.732879    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx5hl\" (UniqueName: \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.733116 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.733066    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.733230 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.733183    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.734137 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.734110    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.795379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.795305    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86lsm\" (UniqueName: \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\") pod \"payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.827355 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.827309    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:41.833870 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.833844    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx5hl\" (UniqueName: \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.834009 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.833992    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.834684 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.834663    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.896331 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.896248    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx5hl\" (UniqueName: \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\") pod \"payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:41.984979 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:41.984943    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:42.204934 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.204815    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:55:42.207301 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:55:42.207269    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85643cf7_97bc_4397_aa67_76ab294d0f32.slice/crio-273be2db0bbdb721b5ceba32b8056085150a078e05fcca7a1dcb15d8e0ac9c01 WatchSource:0}: Error finding container 273be2db0bbdb721b5ceba32b8056085150a078e05fcca7a1dcb15d8e0ac9c01: Status 404 returned error can't find the container with id 273be2db0bbdb721b5ceba32b8056085150a078e05fcca7a1dcb15d8e0ac9c01
Nov 02 15:55:42.365539 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.365425    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:55:42.368354 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:55:42.368327    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeea31b6_0e9c_483b_986b_9c096aeba199.slice/crio-be3cbe0735882350bff9317c6573811e0f2e4d7d1ed96cdcf58753653b371e58 WatchSource:0}: Error finding container be3cbe0735882350bff9317c6573811e0f2e4d7d1ed96cdcf58753653b371e58: Status 404 returned error can't find the container with id be3cbe0735882350bff9317c6573811e0f2e4d7d1ed96cdcf58753653b371e58
Nov 02 15:55:42.621419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.621317    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" event={"ID":"deea31b6-0e9c-483b-986b-9c096aeba199","Type":"ContainerStarted","Data":"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"}
Nov 02 15:55:42.621419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.621377    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" event={"ID":"deea31b6-0e9c-483b-986b-9c096aeba199","Type":"ContainerStarted","Data":"be3cbe0735882350bff9317c6573811e0f2e4d7d1ed96cdcf58753653b371e58"}
Nov 02 15:55:42.621419 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.621401    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:42.622931 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.622903    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" event={"ID":"85643cf7-97bc-4397-aa67-76ab294d0f32","Type":"ContainerStarted","Data":"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"}
Nov 02 15:55:42.623050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.622948    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" event={"ID":"85643cf7-97bc-4397-aa67-76ab294d0f32","Type":"ContainerStarted","Data":"273be2db0bbdb721b5ceba32b8056085150a078e05fcca7a1dcb15d8e0ac9c01"}
Nov 02 15:55:42.623050 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.622969    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:55:42.736595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.736538    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" podStartSLOduration=1.736521895 podStartE2EDuration="1.736521895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:55:42.736343829 +0000 UTC m=+3549.174245971" watchObservedRunningTime="2026-11-02 15:55:42.736521895 +0000 UTC m=+3549.174424040"
Nov 02 15:55:42.855227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:42.855149    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" podStartSLOduration=1.855128368 podStartE2EDuration="1.855128368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:55:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:55:42.853683951 +0000 UTC m=+3549.291586092" watchObservedRunningTime="2026-11-02 15:55:42.855128368 +0000 UTC m=+3549.293030513"
Nov 02 15:55:43.945768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:43.945731    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:55:43.949467 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:43.949444    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.002023 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.001980    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-a60cf3-serving-cert\""
Nov 02 15:55:44.010941 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.010910    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:55:44.059840 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.059806    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.060048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.059858    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.060048 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.059994    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnlp\" (UniqueName: \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.060174 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.060069    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.161532 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.161491    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.161759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.161568    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnlp\" (UniqueName: \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.161759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.161619    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.161759 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.161705    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.161947 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:55:44.161814    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-serving-cert: secret "maas-api-e2e-worker-single-main-a60cf3-serving-cert" not found
Nov 02 15:55:44.161947 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:55:44.161878    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls podName:4b46230b-6fff-4a23-a12b-ec699ecf9af9 nodeName:}" failed. No retries permitted until 2026-11-02 15:55:44.661856653 +0000 UTC m=+3551.099758787 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls") pod "maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" (UID: "4b46230b-6fff-4a23-a12b-ec699ecf9af9") : secret "maas-api-e2e-worker-single-main-a60cf3-serving-cert" not found
Nov 02 15:55:44.162370 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.162338    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.164241 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.164214    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.222980 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.222899    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnlp\" (UniqueName: \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.668297 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.668184    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.670670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.670646    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:44.863755 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:44.863718    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:45.240769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:45.240739    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:55:45.243661 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:55:45.243633    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b46230b_6fff_4a23_a12b_ec699ecf9af9.slice/crio-bee9bfa0eab83614eefb9817f94c0b5bd917e04103790ac81932aeba8cf30686 WatchSource:0}: Error finding container bee9bfa0eab83614eefb9817f94c0b5bd917e04103790ac81932aeba8cf30686: Status 404 returned error can't find the container with id bee9bfa0eab83614eefb9817f94c0b5bd917e04103790ac81932aeba8cf30686
Nov 02 15:55:45.640280 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:45.640165    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" event={"ID":"4b46230b-6fff-4a23-a12b-ec699ecf9af9","Type":"ContainerStarted","Data":"bee9bfa0eab83614eefb9817f94c0b5bd917e04103790ac81932aeba8cf30686"}
Nov 02 15:55:46.646383 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:46.646340    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" event={"ID":"4b46230b-6fff-4a23-a12b-ec699ecf9af9","Type":"ContainerStarted","Data":"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"}
Nov 02 15:55:46.646772 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:46.646506    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:46.778768 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:46.778652    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" podStartSLOduration=2.553254613 podStartE2EDuration="3.778637178s" totalImagesPullingTime="1.225382565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:55:46.777278841 +0000 UTC m=+3553.215180985" watchObservedRunningTime="2026-11-02 15:55:46.778637178 +0000 UTC m=+3553.216539342"
Nov 02 15:55:52.662289 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:52.662251    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:55:53.631774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:53.631742    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:55:53.631962 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:55:53.631829    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:56:01.755628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.755541    4470 generic.go:356] "Generic (PLEG): container finished" podID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerID="13789cdc4a6bb714cd02e3bfc134919f998305a2f836648466e22a3719ad011f" exitCode=0
Nov 02 15:56:01.755628 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.755612    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerDied","Data":"13789cdc4a6bb714cd02e3bfc134919f998305a2f836648466e22a3719ad011f"}
Nov 02 15:56:01.756097 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.756045    4470 scope.go:122] "RemoveContainer" containerID="13789cdc4a6bb714cd02e3bfc134919f998305a2f836648466e22a3719ad011f"
Nov 02 15:56:01.757281 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.757247    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:56:01.757410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.757336    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:56:01.757410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:01.757385    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 15:56:02.763411 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:02.763373    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerStarted","Data":"3d6bbd53ddb8429b8d11bc798dbb0d08966b1b5b273cea7fde52b9334d936da6"}
Nov 02 15:56:02.974650 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:02.974611    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:56:02.983378 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:02.983334    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 02 15:56:02.983720 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:02.983682    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 02 15:56:03.565574 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:56:03.565536    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:56:03.974216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:03.973721    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 02 15:56:03.974216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:03.973818    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 02 15:56:04.970679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:04.970635    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 02 15:56:04.971008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:04.970979    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 02 15:56:05.834677 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.834642    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:56:05.838267 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.838249    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.908821 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.908774    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-a60cf3\"/\"kube-root-ca.crt\""
Nov 02 15:56:05.929911 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.929881    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnsl2\" (UniqueName: \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.930076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.929920    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.930076 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.930047    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.930154 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.930090    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.930206 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.930178    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.930248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.930210    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:05.943283 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.943261    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-a60cf3\"/\"openshift-service-ca.crt\""
Nov 02 15:56:05.943390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.943271    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-a60cf3\"/\"simulator-single-main-a60cf3-kserve-self-signed-certs\""
Nov 02 15:56:05.943390 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.943348    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-a60cf3\"/\"default-dockercfg-64cwb\""
Nov 02 15:56:05.945827 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.945807    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:56:05.987022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.986986    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 02 15:56:05.987707 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:05.987312    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 02 15:56:06.031255 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031215    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031265    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031324    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsl2\" (UniqueName: \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031349    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031409    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031458 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031437    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031786 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031765    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031854 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031828    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.031913 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.031865    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.033519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.033495    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.033948 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.033928    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.096581 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.096480    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnsl2\" (UniqueName: \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\") pod \"simulator-single-main-a60cf3-kserve-f764fc775-cq78c\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") " pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.149953 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.149919    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:06.555502 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.555475    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:56:06.557714 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:56:06.557684    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0366486_7422_4985_b94b_de5d7ed4549a.slice/crio-541c319b90629484e70f69df40ec30b7772eac7af67eeb3aac274ed7d611bceb WatchSource:0}: Error finding container 541c319b90629484e70f69df40ec30b7772eac7af67eeb3aac274ed7d611bceb: Status 404 returned error can't find the container with id 541c319b90629484e70f69df40ec30b7772eac7af67eeb3aac274ed7d611bceb
Nov 02 15:56:06.594342 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.594304    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:56:06.806973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.806880    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" event={"ID":"e0366486-7422-4985-b94b-de5d7ed4549a","Type":"ContainerStarted","Data":"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"}
Nov 02 15:56:06.806973 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.806925    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" event={"ID":"e0366486-7422-4985-b94b-de5d7ed4549a","Type":"ContainerStarted","Data":"541c319b90629484e70f69df40ec30b7772eac7af67eeb3aac274ed7d611bceb"}
Nov 02 15:56:06.925013 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.924955    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" podStartSLOduration=1.924940442 podStartE2EDuration="1.924940442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:56:06.923782529 +0000 UTC m=+3573.361684706" watchObservedRunningTime="2026-11-02 15:56:06.924940442 +0000 UTC m=+3573.362842582"
Nov 02 15:56:06.983620 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.983576    4470 patch_prober.go:29] interesting pod/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused" start-of-body=
Nov 02 15:56:06.983903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:06.983872    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.151:15021/healthz/ready\": dial tcp 10.133.0.151:15021: connect: connection refused"
Nov 02 15:56:07.150919 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:07.150839    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:07.974997 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:07.974966    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:56:07.975903 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:07.975881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:56:16.150438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:16.150400    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:16.163703 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:16.163672    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:16.868744 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:16.868713    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:56:33.565183 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:56:33.565138    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:56:34.178045 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:34.178019    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:56:34.227314 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:56:34.227290    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 15:57:03.565769 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:03.565730    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:57:18.355592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:18.355487    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:57:25.364658 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:25.364602    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:57:25.471604 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:25.471564    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:57:30.365170 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:30.365107    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing" containerID="cri-o://5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899" gracePeriod=25
Nov 02 15:57:30.472035 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:30.471968    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing" containerID="cri-o://f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848" gracePeriod=25
Nov 02 15:57:30.717036 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:30.717000    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:57:30.717264 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:30.717241    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" podUID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" containerName="maas-api" containerID="cri-o://09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40" gracePeriod=30
Nov 02 15:57:31.970726 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:31.970693    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:57:32.050979 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.050884    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") pod \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") "
Nov 02 15:57:32.051151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.051001    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\") pod \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") "
Nov 02 15:57:32.051151 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.051065    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\") pod \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") "
Nov 02 15:57:32.051312 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.051157    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\" (UniqueName: \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\") pod \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\" (UID: \"4b46230b-6fff-4a23-a12b-ec699ecf9af9\") "
Nov 02 15:57:32.051478 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.051446    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca" pod "4b46230b-6fff-4a23-a12b-ec699ecf9af9" (UID: "4b46230b-6fff-4a23-a12b-ec699ecf9af9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:57:32.051625 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.051571    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b46230b-6fff-4a23-a12b-ec699ecf9af9-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:32.053326 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.053302    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp" pod "4b46230b-6fff-4a23-a12b-ec699ecf9af9" (UID: "4b46230b-6fff-4a23-a12b-ec699ecf9af9"). InnerVolumeSpecName "kube-api-access-5cnlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:32.053769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.053741    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls" pod "4b46230b-6fff-4a23-a12b-ec699ecf9af9" (UID: "4b46230b-6fff-4a23-a12b-ec699ecf9af9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:57:32.053846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.053753    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs" pod "4b46230b-6fff-4a23-a12b-ec699ecf9af9" (UID: "4b46230b-6fff-4a23-a12b-ec699ecf9af9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:57:32.152584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.152544    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cnlp\" (UniqueName: \"kubernetes.io/projected/4b46230b-6fff-4a23-a12b-ec699ecf9af9-kube-api-access-5cnlp\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:32.152584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.152578    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:32.152584 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.152587    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b46230b-6fff-4a23-a12b-ec699ecf9af9-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:32.245043 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.245005    4470 generic.go:356] "Generic (PLEG): container finished" podID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" containerID="09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40" exitCode=0
Nov 02 15:57:32.245245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.245053    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"
Nov 02 15:57:32.245245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.245089    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" event={"ID":"4b46230b-6fff-4a23-a12b-ec699ecf9af9","Type":"ContainerDied","Data":"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"}
Nov 02 15:57:32.245245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.245146    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5" event={"ID":"4b46230b-6fff-4a23-a12b-ec699ecf9af9","Type":"ContainerDied","Data":"bee9bfa0eab83614eefb9817f94c0b5bd917e04103790ac81932aeba8cf30686"}
Nov 02 15:57:32.245245 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.245168    4470 scope.go:122] "RemoveContainer" containerID="09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"
Nov 02 15:57:32.255069 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.255049    4470 scope.go:122] "RemoveContainer" containerID="09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"
Nov 02 15:57:32.255403 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:32.255379    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40\": container with ID starting with 09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40 not found: ID does not exist" containerID="09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"
Nov 02 15:57:32.255462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.255415    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40"} err="failed to get container status \"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40\": rpc error: code = NotFound desc = could not find container \"09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40\": container with ID starting with 09f93d31ed30a704d7da1ebd191eace04bfc57bcc3b49320fba872729b172e40 not found: ID does not exist"
Nov 02 15:57:32.417495 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.417384    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:57:32.419692 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:32.419658    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a60cf3-85f5f9f675-4zwj5"]
Nov 02 15:57:33.376042 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.375992    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" podUID="e0366486-7422-4985-b94b-de5d7ed4549a" containerName="main" containerID="cri-o://0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960" gracePeriod=45
Nov 02 15:57:33.568744 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:33.568693    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:57:33.630438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.630365    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:57:33.630438 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.630391    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:57:33.638760 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.638736    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:57:33.705481 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.705446    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" path="/var/lib/kubelet/pods/4b46230b-6fff-4a23-a12b-ec699ecf9af9/volumes"
Nov 02 15:57:33.771172 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771132    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\" (UniqueName: \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771232    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771271    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771386 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771311    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771511    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\" (UniqueName: \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771545    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\") pod \"e0366486-7422-4985-b94b-de5d7ed4549a\" (UID: \"e0366486-7422-4985-b94b-de5d7ed4549a\") "
Nov 02 15:57:33.771696 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771575    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:33.771757 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771739    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:33.772008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.771981    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:33.772008 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.772009    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:33.772321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.772285    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:33.774364 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.774324    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "kube-api-access-fnsl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:33.774500 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.774369    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:57:33.774645 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.774624    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm" pod "e0366486-7422-4985-b94b-de5d7ed4549a" (UID: "e0366486-7422-4985-b94b-de5d7ed4549a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:33.873047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.873011    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnsl2\" (UniqueName: \"kubernetes.io/projected/e0366486-7422-4985-b94b-de5d7ed4549a-kube-api-access-fnsl2\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:33.873047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.873045    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:33.873047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.873056    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0366486-7422-4985-b94b-de5d7ed4549a-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:33.873321 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:33.873067    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e0366486-7422-4985-b94b-de5d7ed4549a-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:34.257450 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.257411    4470 generic.go:356] "Generic (PLEG): container finished" podID="e0366486-7422-4985-b94b-de5d7ed4549a" containerID="0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960" exitCode=0
Nov 02 15:57:34.257630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.257457    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"
Nov 02 15:57:34.257630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.257489    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" event={"ID":"e0366486-7422-4985-b94b-de5d7ed4549a","Type":"ContainerDied","Data":"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"}
Nov 02 15:57:34.257630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.257536    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c" event={"ID":"e0366486-7422-4985-b94b-de5d7ed4549a","Type":"ContainerDied","Data":"541c319b90629484e70f69df40ec30b7772eac7af67eeb3aac274ed7d611bceb"}
Nov 02 15:57:34.257630 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.257553    4470 scope.go:122] "RemoveContainer" containerID="0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"
Nov 02 15:57:34.267550 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.267529    4470 scope.go:122] "RemoveContainer" containerID="0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"
Nov 02 15:57:34.267838 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:34.267817    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960\": container with ID starting with 0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960 not found: ID does not exist" containerID="0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"
Nov 02 15:57:34.267894 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.267849    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960"} err="failed to get container status \"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960\": rpc error: code = NotFound desc = could not find container \"0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960\": container with ID starting with 0bda7e4964601efe5603312b159dd764e4927fab52968bfd87f9887c079cd960 not found: ID does not exist"
Nov 02 15:57:34.421407 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.421371    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:57:34.470906 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:34.470867    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-a60cf3/simulator-single-main-a60cf3-kserve-f764fc775-cq78c"]
Nov 02 15:57:35.705270 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:35.705241    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e0366486-7422-4985-b94b-de5d7ed4549a" path="/var/lib/kubelet/pods/e0366486-7422-4985-b94b-de5d7ed4549a/volumes"
Nov 02 15:57:40.109096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:40.109051    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:57:40.109629 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:40.109348    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy" containerID="cri-o://3d6bbd53ddb8429b8d11bc798dbb0d08966b1b5b273cea7fde52b9334d936da6" gracePeriod=30
Nov 02 15:57:43.630096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:43.630049    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:57:43.630096 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:43.630065    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 15:57:45.316966 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.316934    4470 generic.go:356] "Generic (PLEG): container finished" podID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerID="3d6bbd53ddb8429b8d11bc798dbb0d08966b1b5b273cea7fde52b9334d936da6" exitCode=0
Nov 02 15:57:45.317462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.317015    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerDied","Data":"3d6bbd53ddb8429b8d11bc798dbb0d08966b1b5b273cea7fde52b9334d936da6"}
Nov 02 15:57:45.317462 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.317100    4470 scope.go:122] "RemoveContainer" containerID="13789cdc4a6bb714cd02e3bfc134919f998305a2f836648466e22a3719ad011f"
Nov 02 15:57:45.365046 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.365022    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:57:45.395131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395052    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395090    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395131 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395115    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395140    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395161    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395294    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395408    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395409    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:45.395641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395416    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:45.395641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395442    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395519    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:45.395641 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395548    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\") pod \"308d1dc3-c38c-4d5c-933f-7f2789096f51\" (UID: \"308d1dc3-c38c-4d5c-933f-7f2789096f51\") "
Nov 02 15:57:45.395924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395700    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:57:45.395924 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.395712    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:45.396066 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.396041    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.396126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.396072    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.396126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.396091    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.396126 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.396109    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.396323 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.396125    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/308d1dc3-c38c-4d5c-933f-7f2789096f51-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.397622 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.397590    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "kube-api-access-rvtpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:45.397735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.397661    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 15:57:45.397735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.397696    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:45.397735 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.397705    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy" pod "308d1dc3-c38c-4d5c-933f-7f2789096f51" (UID: "308d1dc3-c38c-4d5c-933f-7f2789096f51"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:57:45.497463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.497421    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.497463 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.497458    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.497693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.497486    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvtpb\" (UniqueName: \"kubernetes.io/projected/308d1dc3-c38c-4d5c-933f-7f2789096f51-kube-api-access-rvtpb\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:45.497693 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:45.497500    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/308d1dc3-c38c-4d5c-933f-7f2789096f51-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:46.323769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.323728    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk" event={"ID":"308d1dc3-c38c-4d5c-933f-7f2789096f51","Type":"ContainerDied","Data":"e814e7bc186ec417f3b3d9116d0e5f44c852511481f577f8de42ae26fee10014"}
Nov 02 15:57:46.323769 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.323754    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"
Nov 02 15:57:46.324285 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.323803    4470 scope.go:122] "RemoveContainer" containerID="3d6bbd53ddb8429b8d11bc798dbb0d08966b1b5b273cea7fde52b9334d936da6"
Nov 02 15:57:46.539040 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.539002    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:57:46.588679 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.588592    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-a60cf3-openshift-default-7b84cbdbfcvkmhk"]
Nov 02 15:57:46.941819 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.941789    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:57:46.942251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942233    4470 cpu_manager.go:414] "containerMap: removing container" podUID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" containerName="maas-api"
Nov 02 15:57:46.942251 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942248    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" containerName="maas-api"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942269    4470 cpu_manager.go:414] "containerMap: removing container" podUID="e0366486-7422-4985-b94b-de5d7ed4549a" containerName="main"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942275    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0366486-7422-4985-b94b-de5d7ed4549a" containerName="main"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942284    4470 cpu_manager.go:414] "containerMap: removing container" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942289    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942295    4470 cpu_manager.go:414] "containerMap: removing container" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.942351 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942301    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.942545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942383    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b46230b-6fff-4a23-a12b-ec699ecf9af9" containerName="maas-api"
Nov 02 15:57:46.942545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942393    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0366486-7422-4985-b94b-de5d7ed4549a" containerName="main"
Nov 02 15:57:46.942545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942399    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.942545 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.942405    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" containerName="istio-proxy"
Nov 02 15:57:46.946791 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.946774    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:46.999087 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:46.999056    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-102aae2b-kserve-self-signed-certs\""
Nov 02 15:57:47.005221 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.005177    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:57:47.014327 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014298    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.014494 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014360    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.014552 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014517    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.014595 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014576    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.014644 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014602    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.014721 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.014707    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt795\" (UniqueName: \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116110    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116217    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116261    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116331    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116379 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116358    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116606 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116414    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt795\" (UniqueName: \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.116776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.116745    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.117047 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.117017    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.117155 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.117059    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.119325 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.119301    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.119670 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.119647    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.178029 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.177995    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt795\" (UniqueName: \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\") pod \"e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") " pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.258942 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.258846    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:47.637558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.637529    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:57:47.640228 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:57:47.640184    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddfa2cd00_c630_4335_82bc_12b5dd02aa66.slice/crio-45970b86a12fc95e5638180e2ec4d061c4486951b8f8287bc11dec0d929feeae WatchSource:0}: Error finding container 45970b86a12fc95e5638180e2ec4d061c4486951b8f8287bc11dec0d929feeae: Status 404 returned error can't find the container with id 45970b86a12fc95e5638180e2ec4d061c4486951b8f8287bc11dec0d929feeae
Nov 02 15:57:47.705586 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:47.705555    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="308d1dc3-c38c-4d5c-933f-7f2789096f51" path="/var/lib/kubelet/pods/308d1dc3-c38c-4d5c-933f-7f2789096f51/volumes"
Nov 02 15:57:48.344275 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:48.344235    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" event={"ID":"dfa2cd00-c630-4335-82bc-12b5dd02aa66","Type":"ContainerStarted","Data":"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"}
Nov 02 15:57:48.344446 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:48.344287    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" event={"ID":"dfa2cd00-c630-4335-82bc-12b5dd02aa66","Type":"ContainerStarted","Data":"45970b86a12fc95e5638180e2ec4d061c4486951b8f8287bc11dec0d929feeae"}
Nov 02 15:57:48.461883 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:48.461811    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" podStartSLOduration=2.4617920079999998 podStartE2EDuration="2.461792008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:57:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:57:48.460832884 +0000 UTC m=+3674.898735027" watchObservedRunningTime="2026-11-02 15:57:48.461792008 +0000 UTC m=+3674.899694149"
Nov 02 15:57:49.259862 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:49.259773    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:50.630964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.630897    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:57:50.655846 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.655808    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\") pod \"85643cf7-97bc-4397-aa67-76ab294d0f32\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") "
Nov 02 15:57:50.655996 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.655926    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\" (UniqueName: \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\") pod \"85643cf7-97bc-4397-aa67-76ab294d0f32\" (UID: \"85643cf7-97bc-4397-aa67-76ab294d0f32\") "
Nov 02 15:57:50.656265 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.656237    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume" pod "85643cf7-97bc-4397-aa67-76ab294d0f32" (UID: "85643cf7-97bc-4397-aa67-76ab294d0f32"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:57:50.658296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.658261    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm" pod "85643cf7-97bc-4397-aa67-76ab294d0f32" (UID: "85643cf7-97bc-4397-aa67-76ab294d0f32"). InnerVolumeSpecName "kube-api-access-86lsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:50.713642 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.713618    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:57:50.757429 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.757396    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\") pod \"deea31b6-0e9c-483b-986b-9c096aeba199\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") "
Nov 02 15:57:50.757599 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.757567    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\" (UniqueName: \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\") pod \"deea31b6-0e9c-483b-986b-9c096aeba199\" (UID: \"deea31b6-0e9c-483b-986b-9c096aeba199\") "
Nov 02 15:57:50.757827 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.757794    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume" pod "deea31b6-0e9c-483b-986b-9c096aeba199" (UID: "deea31b6-0e9c-483b-986b-9c096aeba199"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 15:57:50.758022 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.758000    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85643cf7-97bc-4397-aa67-76ab294d0f32-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:50.758110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.758028    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/deea31b6-0e9c-483b-986b-9c096aeba199-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:50.758110 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.758044    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86lsm\" (UniqueName: \"kubernetes.io/projected/85643cf7-97bc-4397-aa67-76ab294d0f32-kube-api-access-86lsm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:50.759719 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.759698    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl" pod "deea31b6-0e9c-483b-986b-9c096aeba199" (UID: "deea31b6-0e9c-483b-986b-9c096aeba199"). InnerVolumeSpecName "kube-api-access-bx5hl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:57:50.859589 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:50.859554    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx5hl\" (UniqueName: \"kubernetes.io/projected/deea31b6-0e9c-483b-986b-9c096aeba199-kube-api-access-bx5hl\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:57:51.359818 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.359781    4470 generic.go:356] "Generic (PLEG): container finished" podID="deea31b6-0e9c-483b-986b-9c096aeba199" containerID="f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848" exitCode=0
Nov 02 15:57:51.360000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.359831    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"
Nov 02 15:57:51.360000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.359868    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" event={"ID":"deea31b6-0e9c-483b-986b-9c096aeba199","Type":"ContainerDied","Data":"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"}
Nov 02 15:57:51.360000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.359928    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz" event={"ID":"deea31b6-0e9c-483b-986b-9c096aeba199","Type":"ContainerDied","Data":"be3cbe0735882350bff9317c6573811e0f2e4d7d1ed96cdcf58753653b371e58"}
Nov 02 15:57:51.360000 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.359951    4470 scope.go:122] "RemoveContainer" containerID="f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"
Nov 02 15:57:51.361437 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.361410    4470 generic.go:356] "Generic (PLEG): container finished" podID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerID="5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899" exitCode=0
Nov 02 15:57:51.361671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.361464    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"
Nov 02 15:57:51.361671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.361476    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" event={"ID":"85643cf7-97bc-4397-aa67-76ab294d0f32","Type":"ContainerDied","Data":"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"}
Nov 02 15:57:51.361671 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.361519    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh" event={"ID":"85643cf7-97bc-4397-aa67-76ab294d0f32","Type":"ContainerDied","Data":"273be2db0bbdb721b5ceba32b8056085150a078e05fcca7a1dcb15d8e0ac9c01"}
Nov 02 15:57:51.372258 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.371984    4470 scope.go:122] "RemoveContainer" containerID="f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"
Nov 02 15:57:51.372447 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:51.372423    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848\": container with ID starting with f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848 not found: ID does not exist" containerID="f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"
Nov 02 15:57:51.372565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.372458    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848"} err="failed to get container status \"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848\": rpc error: code = NotFound desc = could not find container \"f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848\": container with ID starting with f2faa509982dbda46f9bcd841763eba1c9da020d35deadd7b546641e2af8b848 not found: ID does not exist"
Nov 02 15:57:51.372565 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.372480    4470 scope.go:122] "RemoveContainer" containerID="5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"
Nov 02 15:57:51.388227 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.388210    4470 scope.go:122] "RemoveContainer" containerID="5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"
Nov 02 15:57:51.388467 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:57:51.388448    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899\": container with ID starting with 5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899 not found: ID does not exist" containerID="5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"
Nov 02 15:57:51.388519 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.388475    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899"} err="failed to get container status \"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899\": rpc error: code = NotFound desc = could not find container \"5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899\": container with ID starting with 5e280493410c606cbf03c35609477e0e7b1d11035da6596e5231b0d96af2b899 not found: ID does not exist"
Nov 02 15:57:51.523461 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.523427    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:57:51.529415 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.529391    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-a60cf3-7c644dd866wsnh"]
Nov 02 15:57:51.639666 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.639579    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:57:51.644282 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.644255    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-a60cf3-5686btfdzz"]
Nov 02 15:57:51.705587 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.705555    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" path="/var/lib/kubelet/pods/85643cf7-97bc-4397-aa67-76ab294d0f32/volumes"
Nov 02 15:57:51.705956 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:51.705942    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" path="/var/lib/kubelet/pods/deea31b6-0e9c-483b-986b-9c096aeba199/volumes"
Nov 02 15:57:57.259434 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:57.259396    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:57.272296 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:57.272268    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:57:57.468776 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:57:57.468749    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:58:03.564260 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:58:03.564216    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:58:29.590524 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.590486    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591015    4470 cpu_manager.go:414] "containerMap: removing container" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing"
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591034    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing"
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591106    4470 cpu_manager.go:414] "containerMap: removing container" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing"
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591115    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing"
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591239    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="deea31b6-0e9c-483b-986b-9c096aeba199" containerName="payload-pre-processing"
Nov 02 15:58:29.591266 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.591260    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="85643cf7-97bc-4397-aa67-76ab294d0f32" containerName="payload-processing"
Nov 02 15:58:29.594601 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.594579    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.696548 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.696508    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-102aae2b-kserve-self-signed-certs\""
Nov 02 15:58:29.703410 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703376    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.703591 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703445    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.703591 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703540    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.703591 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703583    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.703745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703656    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwc5q\" (UniqueName: \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.703745 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.703716    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.705210 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.705176    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:58:29.805163 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805122    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805230    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805264    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805298    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805339    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwc5q\" (UniqueName: \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805392 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805375    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805688 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805628    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805695    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.805754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.805741    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.807558 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.807526    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.807811 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.807794    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.866580 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.866499    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwc5q\" (UniqueName: \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\") pod \"e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") " pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:29.906964 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:29.906929    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:30.298015 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:30.297938    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:58:30.300478 ip-10-0-139-197 kubenswrapper[4470]: W1102 15:58:30.300444    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd88d6529_c30c_4211_91d7_186b0c234305.slice/crio-8788ae5733f8ae78704271211da481234a1cf64c70d7532df9b07e8efdce0c81 WatchSource:0}: Error finding container 8788ae5733f8ae78704271211da481234a1cf64c70d7532df9b07e8efdce0c81: Status 404 returned error can't find the container with id 8788ae5733f8ae78704271211da481234a1cf64c70d7532df9b07e8efdce0c81
Nov 02 15:58:30.626905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:30.626812    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" event={"ID":"d88d6529-c30c-4211-91d7-186b0c234305","Type":"ContainerStarted","Data":"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"}
Nov 02 15:58:30.626905 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:30.626866    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" event={"ID":"d88d6529-c30c-4211-91d7-186b0c234305","Type":"ContainerStarted","Data":"8788ae5733f8ae78704271211da481234a1cf64c70d7532df9b07e8efdce0c81"}
Nov 02 15:58:30.744738 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:30.744681    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" podStartSLOduration=1.7446663839999998 podStartE2EDuration="1.744666384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 15:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 15:58:30.743409218 +0000 UTC m=+3717.181311359" watchObservedRunningTime="2026-11-02 15:58:30.744666384 +0000 UTC m=+3717.182568526"
Nov 02 15:58:30.907465 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:30.907378    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:33.563852 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:58:33.563811    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:58:39.907919 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:39.907881    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:39.920468 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:39.920435    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:58:40.703841 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:58:40.703804    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:59:03.565047 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:59:03.565003    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:59:17.757147 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:17.757101    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:59:22.544779 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:22.544737    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:59:32.777171 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:32.777110    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" podUID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" containerName="main" containerID="cri-o://1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259" gracePeriod=45
Nov 02 15:59:33.037485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.037416    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:59:33.040127 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.040096    4470 generic.go:356] "Generic (PLEG): container finished" podID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" containerID="1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259" exitCode=0
Nov 02 15:59:33.040291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.040148    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"
Nov 02 15:59:33.040291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.040169    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" event={"ID":"dfa2cd00-c630-4335-82bc-12b5dd02aa66","Type":"ContainerDied","Data":"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"}
Nov 02 15:59:33.040291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.040230    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc" event={"ID":"dfa2cd00-c630-4335-82bc-12b5dd02aa66","Type":"ContainerDied","Data":"45970b86a12fc95e5638180e2ec4d061c4486951b8f8287bc11dec0d929feeae"}
Nov 02 15:59:33.040291 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.040247    4470 scope.go:122] "RemoveContainer" containerID="1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"
Nov 02 15:59:33.048762 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.048742    4470 scope.go:122] "RemoveContainer" containerID="1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"
Nov 02 15:59:33.049065 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:59:33.049032    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259\": container with ID starting with 1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259 not found: ID does not exist" containerID="1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"
Nov 02 15:59:33.049148 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.049080    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259"} err="failed to get container status \"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259\": rpc error: code = NotFound desc = could not find container \"1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259\": container with ID starting with 1c95523dbf00b49e6d818c0daa93bc14a76b7f27033c0dab179c38a112a6d259 not found: ID does not exist"
Nov 02 15:59:33.176754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.176720    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.176972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.176773    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\" (UniqueName: \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.176972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.176813    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.176972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.176856    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.176972 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.176908    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.177242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177021    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\") pod \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\" (UID: \"dfa2cd00-c630-4335-82bc-12b5dd02aa66\") "
Nov 02 15:59:33.177242 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177139    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:33.177368 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177344    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:33.177592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177503    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.177592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177555    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.177592 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.177581    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:33.179774 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.179731    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:33.179888 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.179861    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "kube-api-access-vt795". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:59:33.179937 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.179925    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs" pod "dfa2cd00-c630-4335-82bc-12b5dd02aa66" (UID: "dfa2cd00-c630-4335-82bc-12b5dd02aa66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:59:33.279014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.278977    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.279014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.279012    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt795\" (UniqueName: \"kubernetes.io/projected/dfa2cd00-c630-4335-82bc-12b5dd02aa66-kube-api-access-vt795\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.279014 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.279022    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dfa2cd00-c630-4335-82bc-12b5dd02aa66-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.279300 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.279032    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dfa2cd00-c630-4335-82bc-12b5dd02aa66-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:33.461228 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.461181    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:59:33.464754 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.464721    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-102aae2b-kserve-79c748db97-xmzsc"]
Nov 02 15:59:33.564853 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:59:33.564814    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 15:59:33.705523 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:33.705487    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" path="/var/lib/kubelet/pods/dfa2cd00-c630-4335-82bc-12b5dd02aa66/volumes"
Nov 02 15:59:37.576563 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.576491    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" podUID="d88d6529-c30c-4211-91d7-186b0c234305" containerName="main" containerID="cri-o://30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680" gracePeriod=45
Nov 02 15:59:37.693445 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.693354    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" podUID="d88d6529-c30c-4211-91d7-186b0c234305" containerName="main" probeResult="failure" output="Get \"https://10.133.0.157:8000/ready\": dial tcp 10.133.0.157:8000: connect: connection refused"
Nov 02 15:59:37.838420 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.838345    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:59:37.929704 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929657    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\" (UniqueName: \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.929896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929715    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.929896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929776    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.929896 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929883    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.930073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929931    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.930073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929977    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\" (UniqueName: \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\") pod \"d88d6529-c30c-4211-91d7-186b0c234305\" (UID: \"d88d6529-c30c-4211-91d7-186b0c234305\") "
Nov 02 15:59:37.930073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.929986    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:37.930073 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.930012    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:37.930319 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.930246    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:37.930426 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.930407    4470 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-tmp-dir\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:37.930485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.930431    4470 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-home\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:37.930485 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.930448    4470 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-model-cache\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:37.932098 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.932058    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 15:59:37.932248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.932122    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 15:59:37.932248 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:37.932168    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q" pod "d88d6529-c30c-4211-91d7-186b0c234305" (UID: "d88d6529-c30c-4211-91d7-186b0c234305"). InnerVolumeSpecName "kube-api-access-gwc5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 15:59:38.031868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.031825    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwc5q\" (UniqueName: \"kubernetes.io/projected/d88d6529-c30c-4211-91d7-186b0c234305-kube-api-access-gwc5q\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:38.031868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.031858    4470 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d88d6529-c30c-4211-91d7-186b0c234305-dshm\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:38.031868 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.031870    4470 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d88d6529-c30c-4211-91d7-186b0c234305-tls-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 15:59:38.067907 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.067871    4470 generic.go:356] "Generic (PLEG): container finished" podID="d88d6529-c30c-4211-91d7-186b0c234305" containerID="30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680" exitCode=0
Nov 02 15:59:38.068095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.067956    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" event={"ID":"d88d6529-c30c-4211-91d7-186b0c234305","Type":"ContainerDied","Data":"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"}
Nov 02 15:59:38.068095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.067975    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"
Nov 02 15:59:38.068095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.067991    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw" event={"ID":"d88d6529-c30c-4211-91d7-186b0c234305","Type":"ContainerDied","Data":"8788ae5733f8ae78704271211da481234a1cf64c70d7532df9b07e8efdce0c81"}
Nov 02 15:59:38.068095 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.068013    4470 scope.go:122] "RemoveContainer" containerID="30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"
Nov 02 15:59:38.081486 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.081458    4470 scope.go:122] "RemoveContainer" containerID="30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"
Nov 02 15:59:38.081855 ip-10-0-139-197 kubenswrapper[4470]: E1102 15:59:38.081836    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680\": container with ID starting with 30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680 not found: ID does not exist" containerID="30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"
Nov 02 15:59:38.081944 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.081865    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680"} err="failed to get container status \"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680\": rpc error: code = NotFound desc = could not find container \"30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680\": container with ID starting with 30de4624efa6627ff297f2faaff1871f8cbf6034b2d0bd67ba6655c89a769680 not found: ID does not exist"
Nov 02 15:59:38.231350 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.231314    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:59:38.237654 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:38.237624    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-102aae2b-kserve-5789b69485-j8ndw"]
Nov 02 15:59:39.705804 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:39.705762    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d88d6529-c30c-4211-91d7-186b0c234305" path="/var/lib/kubelet/pods/d88d6529-c30c-4211-91d7-186b0c234305/volumes"
Nov 02 15:59:42.093216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:42.091067    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 15:59:42.093216 ip-10-0-139-197 kubenswrapper[4470]: I1102 15:59:42.091142    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 16:00:03.564498 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:00:03.564453    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:00:23.449090 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449007    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"]
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449462    4470 cpu_manager.go:414] "containerMap: removing container" podUID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" containerName="main"
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449475    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" containerName="main"
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449487    4470 cpu_manager.go:414] "containerMap: removing container" podUID="d88d6529-c30c-4211-91d7-186b0c234305" containerName="main"
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449492    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d88d6529-c30c-4211-91d7-186b0c234305" containerName="main"
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449566    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="dfa2cd00-c630-4335-82bc-12b5dd02aa66" containerName="main"
Nov 02 16:00:23.449618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.449576    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="d88d6529-c30c-4211-91d7-186b0c234305" containerName="main"
Nov 02 16:00:23.452682 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.452662    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.556499 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.556462    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"]
Nov 02 16:00:23.616426 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.616389    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea565e18-23df-4e7a-8443-df50f4767c56-openshift-service-ca\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.616649 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.616439    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbr7j\" (UniqueName: \"kubernetes.io/projected/ea565e18-23df-4e7a-8443-df50f4767c56-kube-api-access-tbr7j\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.616649 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.616553    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-maas-api-tls\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.616649 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.616614    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-metrics-certs\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.717719 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.717627    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea565e18-23df-4e7a-8443-df50f4767c56-openshift-service-ca\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.717719 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.717691    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbr7j\" (UniqueName: \"kubernetes.io/projected/ea565e18-23df-4e7a-8443-df50f4767c56-kube-api-access-tbr7j\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.717956 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.717729    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-maas-api-tls\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.717956 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.717778    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-metrics-certs\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.718375 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.718349    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea565e18-23df-4e7a-8443-df50f4767c56-openshift-service-ca\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.720399 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.720371    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-metrics-certs\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.720505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.720380    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea565e18-23df-4e7a-8443-df50f4767c56-maas-api-tls\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:23.779262 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:23.779221    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbr7j\" (UniqueName: \"kubernetes.io/projected/ea565e18-23df-4e7a-8443-df50f4767c56-kube-api-access-tbr7j\") pod \"maas-api-d95f6c575-q7vqg\" (UID: \"ea565e18-23df-4e7a-8443-df50f4767c56\") " pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:24.065701 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:24.065607    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:24.446396 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:24.446365    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"]
Nov 02 16:00:24.449123 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:00:24.449091    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea565e18_23df_4e7a_8443_df50f4767c56.slice/crio-ed5c3f08dde6aa6f6c29919a2a72092aa709e077f27a4c653e28fe41ce807eae WatchSource:0}: Error finding container ed5c3f08dde6aa6f6c29919a2a72092aa709e077f27a4c653e28fe41ce807eae: Status 404 returned error can't find the container with id ed5c3f08dde6aa6f6c29919a2a72092aa709e077f27a4c653e28fe41ce807eae
Nov 02 16:00:24.450527 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:24.450493    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 16:00:25.422681 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:25.422640    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg" event={"ID":"ea565e18-23df-4e7a-8443-df50f4767c56","Type":"ContainerStarted","Data":"735b673e1b60fa2a21bda81eed4735cd358da689de1a5708233351940aa1d955"}
Nov 02 16:00:25.422902 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:25.422702    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg" event={"ID":"ea565e18-23df-4e7a-8443-df50f4767c56","Type":"ContainerStarted","Data":"ed5c3f08dde6aa6f6c29919a2a72092aa709e077f27a4c653e28fe41ce807eae"}
Nov 02 16:00:25.422902 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:25.422727    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:25.580614 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:25.580487    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg" podStartSLOduration=1.731768512 podStartE2EDuration="2.580472046s" totalImagesPullingTime="848.703534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:00:25.579869837 +0000 UTC m=+3832.017771978" watchObservedRunningTime="2026-11-02 16:00:25.580472046 +0000 UTC m=+3832.018374187"
Nov 02 16:00:27.802328 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:27.802277    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" podUID="6e7f87aa5e9c9e37d740f6ee1d873224" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 16:00:30.901699 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.900528    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:00:30.907028 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.906991    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.963035 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.962998    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5315ece5-openshift-default-dockercfg-sbd7d\""
Nov 02 16:00:30.993589 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993549    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993589 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993595    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993624    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993731    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993758    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993788    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cdms\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.993861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993808    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.994144 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993901    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:30.994144 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:30.993924    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.015451 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.015413    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:00:31.095083 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095046    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095083 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095086    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095116    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095142    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095176    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095245    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cdms\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095271    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095303    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095337    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095688 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095575    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095764 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095740    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095820 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095774    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095820 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095786    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.095926 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.095843    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.097662 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.097635    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.097830 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.097816    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.157851 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.157774    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.158089 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.158059    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cdms\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\") pod \"e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") " pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.221356 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.221308    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:31.433638 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.433616    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d95f6c575-q7vqg"
Nov 02 16:00:31.653366 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.653333    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:00:31.655581 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:00:31.655553    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077aaaa6_2963_47ab_b830_b7907f1ad378.slice/crio-1dae23c90c015e249b6b9b287d7ea1d2403336967cb56e3df3c7ae880622692e WatchSource:0}: Error finding container 1dae23c90c015e249b6b9b287d7ea1d2403336967cb56e3df3c7ae880622692e: Status 404 returned error can't find the container with id 1dae23c90c015e249b6b9b287d7ea1d2403336967cb56e3df3c7ae880622692e
Nov 02 16:00:31.657802 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.657771    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:31.657866 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.657841    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:31.657907 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:31.657868    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:32.458742 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:32.458697    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerStarted","Data":"877e5d9616118d988aba31772bd4fd4a7fd45b79c6a6421eceea28c886582bf8"}
Nov 02 16:00:32.459167 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:32.458764    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerStarted","Data":"1dae23c90c015e249b6b9b287d7ea1d2403336967cb56e3df3c7ae880622692e"}
Nov 02 16:00:32.631435 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:32.631377    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podStartSLOduration=2.63136163 podStartE2EDuration="2.63136163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:00:32.629056415 +0000 UTC m=+3839.066958558" watchObservedRunningTime="2026-11-02 16:00:32.63136163 +0000 UTC m=+3839.069263771"
Nov 02 16:00:33.221723 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:33.221680    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:33.227795 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:33.227759    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:33.464317 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:33.464284    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:33.465301 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:33.465281    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:33.569144 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:00:33.569056    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:00:38.467828 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.467792    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:00:38.472996 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.472973    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.529337 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.528457    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5315ece5\""
Nov 02 16:00:38.576485 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.576446    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.576669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.576650    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqd2t\" (UniqueName: \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.577304 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.577277    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5315ece5-dockercfg-x7kpk\""
Nov 02 16:00:38.579958 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.579935    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:00:38.595112 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.595068    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:00:38.598829 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.598809    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.630922 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.630889    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:00:38.634428 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.634408    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.650944 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.650919    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5315ece5-serving-cert\""
Nov 02 16:00:38.659814 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.659786    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:00:38.668777 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.668597    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:00:38.677596 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677564    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqd2t\" (UniqueName: \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.677773 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677608    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.677773 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677665    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.677887 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677778    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxhx\" (UniqueName: \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.677926 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677902    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpxj5\" (UniqueName: \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.677959 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677929    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.677959 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.677949    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.678078 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.678051    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.678509 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.678492    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.739133 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.739053    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqd2t\" (UniqueName: \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\") pod \"payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.779698 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.779656    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxhx\" (UniqueName: \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.779904 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.779750    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpxj5\" (UniqueName: \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.779904 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.779796    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.779904 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.779854    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.780099 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.779937    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.780099 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.780020    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.780216 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:00:38.780179    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-serving-cert: secret "maas-api-e2e-ait-5315ece5-serving-cert" not found
Nov 02 16:00:38.780281 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:00:38.780269    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls podName:94eaa0b6-7c6f-4178-92b9-3c15070927c9 nodeName:}" failed. No retries permitted until 2026-11-02 16:00:39.280249005 +0000 UTC m=+3845.718151129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls") pod "maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" (UID: "94eaa0b6-7c6f-4178-92b9-3c15070927c9") : secret "maas-api-e2e-ait-5315ece5-serving-cert" not found
Nov 02 16:00:38.780805 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.780780    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.780805 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.780801    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.782445 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.782425    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.786385 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.786361    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:38.875178 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.871367    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxhx\" (UniqueName: \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:38.875178 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.872694    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpxj5\" (UniqueName: \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\") pod \"payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") " pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:38.947558 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:38.947511    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:39.179512 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.179489    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:00:39.181284 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:00:39.181262    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906af184_8484_4b3b_9fff_bcc25252faa8.slice/crio-23bb6c96105c495d65e547cf5acbeeba562c443964d5dcc5c0beaa4486bc4411 WatchSource:0}: Error finding container 23bb6c96105c495d65e547cf5acbeeba562c443964d5dcc5c0beaa4486bc4411: Status 404 returned error can't find the container with id 23bb6c96105c495d65e547cf5acbeeba562c443964d5dcc5c0beaa4486bc4411
Nov 02 16:00:39.285641 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.285560    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:39.288020 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.287987    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") pod \"maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:39.350585 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.350557    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:00:39.352337 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:00:39.352311    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fa643e9_e5e2_4646_a15c_a1486d6fdc40.slice/crio-d272c35a575dd50a05d406669ab5839c640e85c03081bebb5fd94162903a0422 WatchSource:0}: Error finding container d272c35a575dd50a05d406669ab5839c640e85c03081bebb5fd94162903a0422: Status 404 returned error can't find the container with id d272c35a575dd50a05d406669ab5839c640e85c03081bebb5fd94162903a0422
Nov 02 16:00:39.495678 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.495630    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" event={"ID":"906af184-8484-4b3b-9fff-bcc25252faa8","Type":"ContainerStarted","Data":"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"}
Nov 02 16:00:39.496121 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.495707    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" event={"ID":"906af184-8484-4b3b-9fff-bcc25252faa8","Type":"ContainerStarted","Data":"23bb6c96105c495d65e547cf5acbeeba562c443964d5dcc5c0beaa4486bc4411"}
Nov 02 16:00:39.496121 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.495747    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:39.497283 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.497259    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" event={"ID":"9fa643e9-e5e2-4646-a15c-a1486d6fdc40","Type":"ContainerStarted","Data":"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"}
Nov 02 16:00:39.497362 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.497303    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" event={"ID":"9fa643e9-e5e2-4646-a15c-a1486d6fdc40","Type":"ContainerStarted","Data":"d272c35a575dd50a05d406669ab5839c640e85c03081bebb5fd94162903a0422"}
Nov 02 16:00:39.497413 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.497393    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:39.512713 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.512685    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:39.614837 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.614772    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" podStartSLOduration=1.614756842 podStartE2EDuration="1.614756842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:00:39.611907652 +0000 UTC m=+3846.049809794" watchObservedRunningTime="2026-11-02 16:00:39.614756842 +0000 UTC m=+3846.052658983"
Nov 02 16:00:39.730262 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.730171    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" podStartSLOduration=1.730152458 podStartE2EDuration="1.730152458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:00:39.728137008 +0000 UTC m=+3846.166039146" watchObservedRunningTime="2026-11-02 16:00:39.730152458 +0000 UTC m=+3846.168054600"
Nov 02 16:00:39.890890 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.890855    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:00:39.893709 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:00:39.893680    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94eaa0b6_7c6f_4178_92b9_3c15070927c9.slice/crio-0b56552c28971b435f81f58bf6a4c8220b2f3635d9642e2baefd21ed5694de8a WatchSource:0}: Error finding container 0b56552c28971b435f81f58bf6a4c8220b2f3635d9642e2baefd21ed5694de8a: Status 404 returned error can't find the container with id 0b56552c28971b435f81f58bf6a4c8220b2f3635d9642e2baefd21ed5694de8a
Nov 02 16:00:39.901588 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.901556    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 16:00:39.901861 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:39.901809    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" podUID="91b5032f-6be1-400d-b463-3007837e608b" containerName="maas-api" containerID="cri-o://1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788" gracePeriod=30
Nov 02 16:00:40.503777 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:40.503737    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" event={"ID":"94eaa0b6-7c6f-4178-92b9-3c15070927c9","Type":"ContainerStarted","Data":"0b56552c28971b435f81f58bf6a4c8220b2f3635d9642e2baefd21ed5694de8a"}
Nov 02 16:00:41.163342 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.163312    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 16:00:41.203841 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.203806    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\") pod \"91b5032f-6be1-400d-b463-3007837e608b\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") "
Nov 02 16:00:41.204037 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.204018    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\") pod \"91b5032f-6be1-400d-b463-3007837e608b\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") "
Nov 02 16:00:41.204112 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.204059    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\") pod \"91b5032f-6be1-400d-b463-3007837e608b\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") "
Nov 02 16:00:41.204166 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.204111    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\" (UniqueName: \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\") pod \"91b5032f-6be1-400d-b463-3007837e608b\" (UID: \"91b5032f-6be1-400d-b463-3007837e608b\") "
Nov 02 16:00:41.204539 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.204510    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca" pod "91b5032f-6be1-400d-b463-3007837e608b" (UID: "91b5032f-6be1-400d-b463-3007837e608b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:00:41.205948 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.205917    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls" pod "91b5032f-6be1-400d-b463-3007837e608b" (UID: "91b5032f-6be1-400d-b463-3007837e608b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:00:41.206050 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.206016    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs" pod "91b5032f-6be1-400d-b463-3007837e608b" (UID: "91b5032f-6be1-400d-b463-3007837e608b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:00:41.206141 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.206121    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6" pod "91b5032f-6be1-400d-b463-3007837e608b" (UID: "91b5032f-6be1-400d-b463-3007837e608b"). InnerVolumeSpecName "kube-api-access-2zvk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:00:41.305505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.305430    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zvk6\" (UniqueName: \"kubernetes.io/projected/91b5032f-6be1-400d-b463-3007837e608b-kube-api-access-2zvk6\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:00:41.305505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.305463    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:00:41.305505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.305480    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91b5032f-6be1-400d-b463-3007837e608b-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:00:41.305505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.305494    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/91b5032f-6be1-400d-b463-3007837e608b-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:00:41.510317 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.510280    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" event={"ID":"94eaa0b6-7c6f-4178-92b9-3c15070927c9","Type":"ContainerStarted","Data":"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"}
Nov 02 16:00:41.510770 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.510450    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:41.511741 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.511715    4470 generic.go:356] "Generic (PLEG): container finished" podID="91b5032f-6be1-400d-b463-3007837e608b" containerID="1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788" exitCode=0
Nov 02 16:00:41.511879 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.511756    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl"
Nov 02 16:00:41.511879 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.511760    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" event={"ID":"91b5032f-6be1-400d-b463-3007837e608b","Type":"ContainerDied","Data":"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"}
Nov 02 16:00:41.511879 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.511858    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-88df9b747-twfbl" event={"ID":"91b5032f-6be1-400d-b463-3007837e608b","Type":"ContainerDied","Data":"590dba5e87347a19773e25d9f63c4894165b55c8bb888a5b6550a6dddfd0d6e4"}
Nov 02 16:00:41.512043 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.511886    4470 scope.go:122] "RemoveContainer" containerID="1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"
Nov 02 16:00:41.521174 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.521151    4470 scope.go:122] "RemoveContainer" containerID="1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"
Nov 02 16:00:41.521477 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:00:41.521450    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788\": container with ID starting with 1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788 not found: ID does not exist" containerID="1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"
Nov 02 16:00:41.521531 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.521489    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788"} err="failed to get container status \"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788\": rpc error: code = NotFound desc = could not find container \"1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788\": container with ID starting with 1d654135f86c7a3509c0f670be774944d98367e907fdb428170fee5412bc9788 not found: ID does not exist"
Nov 02 16:00:41.630158 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.630060    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" podStartSLOduration=2.794964137 podStartE2EDuration="3.630047062s" totalImagesPullingTime="835.082925ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:00:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:00:41.628568005 +0000 UTC m=+3848.066470149" watchObservedRunningTime="2026-11-02 16:00:41.630047062 +0000 UTC m=+3848.067949203"
Nov 02 16:00:41.740441 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.740404    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 16:00:41.744841 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:41.744811    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-88df9b747-twfbl"]
Nov 02 16:00:43.705025 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:43.704989    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91b5032f-6be1-400d-b463-3007837e608b" path="/var/lib/kubelet/pods/91b5032f-6be1-400d-b463-3007837e608b/volumes"
Nov 02 16:00:47.524452 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:47.524420    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:00:50.507171 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:50.507136    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:00:50.507649 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:50.507334    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:00:56.593969 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.593869    4470 generic.go:356] "Generic (PLEG): container finished" podID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerID="877e5d9616118d988aba31772bd4fd4a7fd45b79c6a6421eceea28c886582bf8" exitCode=0
Nov 02 16:00:56.593969 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.593949    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerDied","Data":"877e5d9616118d988aba31772bd4fd4a7fd45b79c6a6421eceea28c886582bf8"}
Nov 02 16:00:56.594500 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.594438    4470 scope.go:122] "RemoveContainer" containerID="877e5d9616118d988aba31772bd4fd4a7fd45b79c6a6421eceea28c886582bf8"
Nov 02 16:00:56.595741 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.595713    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:56.595820 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.595780    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:56.595820 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:56.595813    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:00:57.601534 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:57.601499    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerStarted","Data":"5f8cc1949d66da6ed7e5d35b8241a9ef084f2b1adb9221a00af2898d6b814980"}
Nov 02 16:00:58.221942 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:58.221898    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:00:58.222143 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:58.222085    4470 patch_prober.go:29] interesting pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 02 16:00:58.222215 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:58.222142    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 02 16:00:59.222548 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:59.222503    4470 patch_prober.go:29] interesting pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 02 16:00:59.223003 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:00:59.222570    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 02 16:01:00.222378 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:00.222335    4470 patch_prober.go:29] interesting pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 02 16:01:00.222604 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:00.222399    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 02 16:01:01.222423 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:01.222379    4470 patch_prober.go:29] interesting pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 02 16:01:01.222618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:01.222457    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 02 16:01:02.222501 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:02.222450    4470 patch_prober.go:29] interesting pod/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused" start-of-body=
Nov 02 16:01:02.222796 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:02.222523    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.159:15021/healthz/ready\": dial tcp 10.133.0.159:15021: connect: connection refused"
Nov 02 16:01:03.226909 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:03.226875    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:01:03.227348 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:03.227299    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:01:03.228066 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:03.228048    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:01:03.567807 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:03.567718    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:01:17.441603 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.441566    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:01:17.442039 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.441782    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerName="maas-api" containerID="cri-o://cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249" gracePeriod=30
Nov 02 16:01:17.518911 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.518858    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.161:8443/health\": dial tcp 10.133.0.161:8443: connect: connection refused"
Nov 02 16:01:17.710612 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.710583    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:01:17.715368 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.715332    4470 generic.go:356] "Generic (PLEG): container finished" podID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerID="cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249" exitCode=0
Nov 02 16:01:17.715527 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.715385    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"
Nov 02 16:01:17.715527 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.715417    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" event={"ID":"94eaa0b6-7c6f-4178-92b9-3c15070927c9","Type":"ContainerDied","Data":"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"}
Nov 02 16:01:17.715527 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.715483    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd" event={"ID":"94eaa0b6-7c6f-4178-92b9-3c15070927c9","Type":"ContainerDied","Data":"0b56552c28971b435f81f58bf6a4c8220b2f3635d9642e2baefd21ed5694de8a"}
Nov 02 16:01:17.715644 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.715519    4470 scope.go:122] "RemoveContainer" containerID="cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"
Nov 02 16:01:17.724214 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.724179    4470 scope.go:122] "RemoveContainer" containerID="cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"
Nov 02 16:01:17.724519 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:17.724491    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249\": container with ID starting with cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249 not found: ID does not exist" containerID="cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"
Nov 02 16:01:17.724578 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.724531    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249"} err="failed to get container status \"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249\": rpc error: code = NotFound desc = could not find container \"cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249\": container with ID starting with cca5036f1bcf9ba52b618b96e15f563cd9f438f65b10f99c158e27fb47e33249 not found: ID does not exist"
Nov 02 16:01:17.827174 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.827139    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\") pod \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") "
Nov 02 16:01:17.827397 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.827247    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\" (UniqueName: \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\") pod \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") "
Nov 02 16:01:17.827397 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.827347    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") pod \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") "
Nov 02 16:01:17.827397 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.827392    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\") pod \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\" (UID: \"94eaa0b6-7c6f-4178-92b9-3c15070927c9\") "
Nov 02 16:01:17.828119 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.827833    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca" pod "94eaa0b6-7c6f-4178-92b9-3c15070927c9" (UID: "94eaa0b6-7c6f-4178-92b9-3c15070927c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:01:17.829897 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.829860    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx" pod "94eaa0b6-7c6f-4178-92b9-3c15070927c9" (UID: "94eaa0b6-7c6f-4178-92b9-3c15070927c9"). InnerVolumeSpecName "kube-api-access-gcxhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:01:17.830295 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.830259    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs" pod "94eaa0b6-7c6f-4178-92b9-3c15070927c9" (UID: "94eaa0b6-7c6f-4178-92b9-3c15070927c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:01:17.830557 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.830515    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls" pod "94eaa0b6-7c6f-4178-92b9-3c15070927c9" (UID: "94eaa0b6-7c6f-4178-92b9-3c15070927c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:01:17.903711 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.903672    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:01:17.928850 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.928812    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:17.928850 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.928845    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcxhx\" (UniqueName: \"kubernetes.io/projected/94eaa0b6-7c6f-4178-92b9-3c15070927c9-kube-api-access-gcxhx\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:17.928850 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.928858    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94eaa0b6-7c6f-4178-92b9-3c15070927c9-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:17.929087 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.928867    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94eaa0b6-7c6f-4178-92b9-3c15070927c9-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:17.974871 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:17.974792    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:01:18.250596 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:18.250516    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:01:18.255181 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:18.255151    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5315ece5-767d8cffc-sqxxd"]
Nov 02 16:01:19.705608 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:19.705568    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" path="/var/lib/kubelet/pods/94eaa0b6-7c6f-4178-92b9-3c15070927c9/volumes"
Nov 02 16:01:22.904230 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:22.904126    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" podUID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" containerName="payload-processing" containerID="cri-o://50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26" gracePeriod=25
Nov 02 16:01:22.975161 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:22.975115    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" podUID="906af184-8484-4b3b-9fff-bcc25252faa8" containerName="payload-pre-processing" containerID="cri-o://5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2" gracePeriod=25
Nov 02 16:01:23.233712 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.233686    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:01:23.256076 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.256054    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:01:23.391495 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.391468    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\" (UniqueName: \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\") pod \"906af184-8484-4b3b-9fff-bcc25252faa8\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") "
Nov 02 16:01:23.391679 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.391576    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\") pod \"906af184-8484-4b3b-9fff-bcc25252faa8\" (UID: \"906af184-8484-4b3b-9fff-bcc25252faa8\") "
Nov 02 16:01:23.391679 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.391653    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\" (UniqueName: \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\") pod \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") "
Nov 02 16:01:23.391679 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.391675    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\") pod \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\" (UID: \"9fa643e9-e5e2-4646-a15c-a1486d6fdc40\") "
Nov 02 16:01:23.392175 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.392140    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume" pod "906af184-8484-4b3b-9fff-bcc25252faa8" (UID: "906af184-8484-4b3b-9fff-bcc25252faa8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:01:23.392319 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.392174    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume" pod "9fa643e9-e5e2-4646-a15c-a1486d6fdc40" (UID: "9fa643e9-e5e2-4646-a15c-a1486d6fdc40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:01:23.393762 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.393734    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t" pod "906af184-8484-4b3b-9fff-bcc25252faa8" (UID: "906af184-8484-4b3b-9fff-bcc25252faa8"). InnerVolumeSpecName "kube-api-access-qqd2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:01:23.393874 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.393814    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5" pod "9fa643e9-e5e2-4646-a15c-a1486d6fdc40" (UID: "9fa643e9-e5e2-4646-a15c-a1486d6fdc40"). InnerVolumeSpecName "kube-api-access-wpxj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:01:23.493244 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.493207    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906af184-8484-4b3b-9fff-bcc25252faa8-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:23.493421 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.493248    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpxj5\" (UniqueName: \"kubernetes.io/projected/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-kube-api-access-wpxj5\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:23.493421 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.493264    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9fa643e9-e5e2-4646-a15c-a1486d6fdc40-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:23.493421 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.493278    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqd2t\" (UniqueName: \"kubernetes.io/projected/906af184-8484-4b3b-9fff-bcc25252faa8-kube-api-access-qqd2t\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:23.746329 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.746236    4470 generic.go:356] "Generic (PLEG): container finished" podID="906af184-8484-4b3b-9fff-bcc25252faa8" containerID="5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2" exitCode=0
Nov 02 16:01:23.746329 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.746285    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"
Nov 02 16:01:23.746557 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.746415    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" event={"ID":"906af184-8484-4b3b-9fff-bcc25252faa8","Type":"ContainerDied","Data":"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"}
Nov 02 16:01:23.746557 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.746471    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq" event={"ID":"906af184-8484-4b3b-9fff-bcc25252faa8","Type":"ContainerDied","Data":"23bb6c96105c495d65e547cf5acbeeba562c443964d5dcc5c0beaa4486bc4411"}
Nov 02 16:01:23.746557 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.746517    4470 scope.go:122] "RemoveContainer" containerID="5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"
Nov 02 16:01:23.747823 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.747803    4470 generic.go:356] "Generic (PLEG): container finished" podID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" containerID="50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26" exitCode=0
Nov 02 16:01:23.747894 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.747846    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" event={"ID":"9fa643e9-e5e2-4646-a15c-a1486d6fdc40","Type":"ContainerDied","Data":"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"}
Nov 02 16:01:23.747936 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.747895    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh" event={"ID":"9fa643e9-e5e2-4646-a15c-a1486d6fdc40","Type":"ContainerDied","Data":"d272c35a575dd50a05d406669ab5839c640e85c03081bebb5fd94162903a0422"}
Nov 02 16:01:23.747990 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.747957    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"
Nov 02 16:01:23.757426 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.757409    4470 scope.go:122] "RemoveContainer" containerID="5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"
Nov 02 16:01:23.757713 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:23.757694    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2\": container with ID starting with 5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2 not found: ID does not exist" containerID="5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"
Nov 02 16:01:23.757713 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.757725    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2"} err="failed to get container status \"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2\": rpc error: code = NotFound desc = could not find container \"5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2\": container with ID starting with 5f5e126403963c4ff54c5f1a8e0b4f973b89edb3b31a037068a6ed48e6ed13c2 not found: ID does not exist"
Nov 02 16:01:23.757713 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.757747    4470 scope.go:122] "RemoveContainer" containerID="50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"
Nov 02 16:01:23.766683 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.766656    4470 scope.go:122] "RemoveContainer" containerID="50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"
Nov 02 16:01:23.766966 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:23.766941    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26\": container with ID starting with 50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26 not found: ID does not exist" containerID="50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"
Nov 02 16:01:23.767031 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.766974    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26"} err="failed to get container status \"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26\": rpc error: code = NotFound desc = could not find container \"50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26\": container with ID starting with 50fca1c5f257a05ce3d0d9d6447b671b84e33831f4027892dfd0b95ba632ed26 not found: ID does not exist"
Nov 02 16:01:23.910738 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.910708    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:01:23.914644 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:23.914620    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5315ece5-778b49bd7f-5tjnh"]
Nov 02 16:01:24.023719 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:24.023628    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:01:24.027958 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:24.027928    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5315ece5-f79f4dfbc-nsnzq"]
Nov 02 16:01:25.705125 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:25.705082    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="906af184-8484-4b3b-9fff-bcc25252faa8" path="/var/lib/kubelet/pods/906af184-8484-4b3b-9fff-bcc25252faa8/volumes"
Nov 02 16:01:25.705564 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:25.705534    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" path="/var/lib/kubelet/pods/9fa643e9-e5e2-4646-a15c-a1486d6fdc40/volumes"
Nov 02 16:01:27.680489 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:27.680452    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:01:27.681035 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:27.680725    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy" containerID="cri-o://5f8cc1949d66da6ed7e5d35b8241a9ef084f2b1adb9221a00af2898d6b814980" gracePeriod=30
Nov 02 16:01:29.543085 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543042    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:01:29.543705 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543683    4470 cpu_manager.go:414] "containerMap: removing container" podUID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" containerName="payload-processing"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543708    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" containerName="payload-processing"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543728    4470 cpu_manager.go:414] "containerMap: removing container" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerName="maas-api"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543734    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerName="maas-api"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543767    4470 cpu_manager.go:414] "containerMap: removing container" podUID="91b5032f-6be1-400d-b463-3007837e608b" containerName="maas-api"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543775    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91b5032f-6be1-400d-b463-3007837e608b" containerName="maas-api"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543791    4470 cpu_manager.go:414] "containerMap: removing container" podUID="906af184-8484-4b3b-9fff-bcc25252faa8" containerName="payload-pre-processing"
Nov 02 16:01:29.543803 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543799    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="906af184-8484-4b3b-9fff-bcc25252faa8" containerName="payload-pre-processing"
Nov 02 16:01:29.544168 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543930    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fa643e9-e5e2-4646-a15c-a1486d6fdc40" containerName="payload-processing"
Nov 02 16:01:29.544168 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543945    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="94eaa0b6-7c6f-4178-92b9-3c15070927c9" containerName="maas-api"
Nov 02 16:01:29.544168 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543958    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="91b5032f-6be1-400d-b463-3007837e608b" containerName="maas-api"
Nov 02 16:01:29.544168 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.543967    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="906af184-8484-4b3b-9fff-bcc25252faa8" containerName="payload-pre-processing"
Nov 02 16:01:29.548642 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.548619    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.612477 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.612366    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-2afef9-openshift-default-dockercfg-j5hnv\""
Nov 02 16:01:29.652145 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652099    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2pbd\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652407 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652161    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652407 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652227    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652407 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652321    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652407 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652368    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652420    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652466    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652491    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.652669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.652537    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.660758 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.660718    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:01:29.753647 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753612    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.753857 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753673    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2pbd\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.753857 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753709    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.753857 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753739    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.753857 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753836    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754098 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753879    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754098 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753931    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754098 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.753994    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754098 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.754021    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754338 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.754255    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754428 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.754406    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754510 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.754490    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.754618 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.754599    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.755034 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.755009    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.756343 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.756324    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.756453 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.756353    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.815012 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.814927    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.815206 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.815095    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2pbd\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\") pod \"e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7\" (UID: \"47566184-c99c-4402-9772-553987d16284\") " pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:29.861316 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:29.861283    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:30.291730 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.291701    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:01:30.294232 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:01:30.294200    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47566184_c99c_4402_9772_553987d16284.slice/crio-b8fbc7ba0d550c083ee9a1c334f7c4e3326749f938763eec7b8a7c7d17a34f85 WatchSource:0}: Error finding container b8fbc7ba0d550c083ee9a1c334f7c4e3326749f938763eec7b8a7c7d17a34f85: Status 404 returned error can't find the container with id b8fbc7ba0d550c083ee9a1c334f7c4e3326749f938763eec7b8a7c7d17a34f85
Nov 02 16:01:30.296659 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.296621    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:30.296751 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.296692    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:30.296751 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.296722    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:30.788092 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.788047    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerStarted","Data":"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"}
Nov 02 16:01:30.788607 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.788114    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerStarted","Data":"b8fbc7ba0d550c083ee9a1c334f7c4e3326749f938763eec7b8a7c7d17a34f85"}
Nov 02 16:01:30.862055 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.862027    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:30.910367 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:30.910314    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podStartSLOduration=1.9102988810000001 podStartE2EDuration="1.910298881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:01:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:01:30.909003824 +0000 UTC m=+3897.346905966" watchObservedRunningTime="2026-11-02 16:01:30.910298881 +0000 UTC m=+3897.348201019"
Nov 02 16:01:31.867814 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:31.867776    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:32.799941 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.799805    4470 generic.go:356] "Generic (PLEG): container finished" podID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerID="5f8cc1949d66da6ed7e5d35b8241a9ef084f2b1adb9221a00af2898d6b814980" exitCode=0
Nov 02 16:01:32.799941 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.799872    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerDied","Data":"5f8cc1949d66da6ed7e5d35b8241a9ef084f2b1adb9221a00af2898d6b814980"}
Nov 02 16:01:32.799941 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.799930    4470 scope.go:122] "RemoveContainer" containerID="877e5d9616118d988aba31772bd4fd4a7fd45b79c6a6421eceea28c886582bf8"
Nov 02 16:01:32.800247 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.800155    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:32.801565 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.801542    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:32.944702 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.944669    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:01:32.989369 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989284    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989369 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989344    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989379    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989437    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989493    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989526    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989562    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989625 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989606    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989951 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989668    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\") pod \"077aaaa6-2963-47ab-b830-b7907f1ad378\" (UID: \"077aaaa6-2963-47ab-b830-b7907f1ad378\") "
Nov 02 16:01:32.989951 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989749    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:01:32.989951 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989830    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:01:32.989951 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989846    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:01:32.990180 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.989993    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:01:32.990180 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.990155    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:32.990324 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.990182    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:32.990324 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.990223    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:32.990324 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.990238    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:32.990324 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.990155    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:01:32.991917 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.991886    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:01:32.992099 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.992081    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:01:32.992172 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.992150    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "kube-api-access-8cdms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:01:32.992240 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:32.992210    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo" pod "077aaaa6-2963-47ab-b830-b7907f1ad378" (UID: "077aaaa6-2963-47ab-b830-b7907f1ad378"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 16:01:33.091297 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.091245    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/077aaaa6-2963-47ab-b830-b7907f1ad378-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:33.091297 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.091297    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cdms\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-kube-api-access-8cdms\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:33.091513 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.091313    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:33.091513 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.091325    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:33.091513 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.091339    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/077aaaa6-2963-47ab-b830-b7907f1ad378-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:01:33.564514 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:33.564477    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:01:33.806296 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.806267    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"
Nov 02 16:01:33.806507 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.806304    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n" event={"ID":"077aaaa6-2963-47ab-b830-b7907f1ad378","Type":"ContainerDied","Data":"1dae23c90c015e249b6b9b287d7ea1d2403336967cb56e3df3c7ae880622692e"}
Nov 02 16:01:33.806507 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.806365    4470 scope.go:122] "RemoveContainer" containerID="5f8cc1949d66da6ed7e5d35b8241a9ef084f2b1adb9221a00af2898d6b814980"
Nov 02 16:01:33.971289 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:33.971253    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:01:34.024680 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:34.024625    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5315ece5-openshift-default-6f776d87df-7wv6n"]
Nov 02 16:01:34.209442 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:34.209415    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 16:01:34.260103 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:34.260074    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 16:01:35.704755 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:35.704726    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" path="/var/lib/kubelet/pods/077aaaa6-2963-47ab-b830-b7907f1ad378/volumes"
Nov 02 16:01:37.709641 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.709602    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:01:37.710346 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.710317    4470 cpu_manager.go:414] "containerMap: removing container" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.710346 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.710344    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.710591 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.710569    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.710741 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.710598    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.715539 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.715518    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:37.768302 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.768257    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:01:37.815427 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.815380    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-2afef9\""
Nov 02 16:01:37.815648 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.815509    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-2afef9-dockercfg-dbzlx\""
Nov 02 16:01:37.841613 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.841579    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnk64\" (UniqueName: \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:37.841816 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.841640    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:37.872081 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.872046    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:01:37.872560 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.872544    4470 cpu_manager.go:414] "containerMap: removing container" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.872560 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.872561    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="077aaaa6-2963-47ab-b830-b7907f1ad378" containerName="istio-proxy"
Nov 02 16:01:37.875856 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.875838    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:37.935774 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.935738    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:01:37.942578 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.942544    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnk64\" (UniqueName: \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:37.942784 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.942616    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:37.942784 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.942644    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:37.942784 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.942734    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq9qj\" (UniqueName: \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:37.943414 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:37.943394    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:38.004381 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.004298    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnk64\" (UniqueName: \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\") pod \"payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:38.029104 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.029062    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:38.043902 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.043866    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.044041 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.043973    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq9qj\" (UniqueName: \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.044627 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.044603    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.106479 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.106451    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq9qj\" (UniqueName: \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\") pod \"payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.189712 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.189672    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.405739 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.405714    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:01:38.570247 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.570205    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:01:38.579849 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:01:38.579817    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc94bfe95_d49d_4942_9bb7_d81ba9dd2215.slice/crio-7f957e0025a79df890bb87218ab82ae68ac46b8562b3dba3a0644302f480cd3f WatchSource:0}: Error finding container 7f957e0025a79df890bb87218ab82ae68ac46b8562b3dba3a0644302f480cd3f: Status 404 returned error can't find the container with id 7f957e0025a79df890bb87218ab82ae68ac46b8562b3dba3a0644302f480cd3f
Nov 02 16:01:38.834739 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.834699    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" event={"ID":"7d502d62-fd41-448c-a2a0-424d4a4f4278","Type":"ContainerStarted","Data":"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"}
Nov 02 16:01:38.835270 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.834753    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" event={"ID":"7d502d62-fd41-448c-a2a0-424d4a4f4278","Type":"ContainerStarted","Data":"c2e2cae6a40cfaa2147454425d446c11915be39840219db81035f31a01f191fe"}
Nov 02 16:01:38.835270 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.834860    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:38.836287 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.836262    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" event={"ID":"c94bfe95-d49d-4942-9bb7-d81ba9dd2215","Type":"ContainerStarted","Data":"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"}
Nov 02 16:01:38.836356 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.836304    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" event={"ID":"c94bfe95-d49d-4942-9bb7-d81ba9dd2215","Type":"ContainerStarted","Data":"7f957e0025a79df890bb87218ab82ae68ac46b8562b3dba3a0644302f480cd3f"}
Nov 02 16:01:38.836356 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.836334    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:38.953382 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:38.953325    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" podStartSLOduration=1.9533097179999999 podStartE2EDuration="1.953309718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:01:38.952525047 +0000 UTC m=+3905.390427188" watchObservedRunningTime="2026-11-02 16:01:38.953309718 +0000 UTC m=+3905.391211860"
Nov 02 16:01:39.070644 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.070589    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" podStartSLOduration=2.070572851 podStartE2EDuration="2.070572851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:01:39.068274139 +0000 UTC m=+3905.506176279" watchObservedRunningTime="2026-11-02 16:01:39.070572851 +0000 UTC m=+3905.508474991"
Nov 02 16:01:39.854756 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.854722    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:01:39.858607 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.858583    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:39.910717 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.910676    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-2afef9-serving-cert\""
Nov 02 16:01:39.962436 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.962402    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:01:39.962926 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.962907    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:39.963045 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.963026    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:39.963255 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.963226    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:39.963338 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:39.963291    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v759\" (UniqueName: \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.064155 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.064114    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.064365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.064221    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.064365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.064254    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v759\" (UniqueName: \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.064365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.064323    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.064365 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:40.064349    4470 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-serving-cert: secret "maas-api-e2e-authcfg-http-2afef9-serving-cert" not found
Nov 02 16:01:40.064585 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:01:40.064411    4470 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls podName:30631da0-b35f-4a13-8033-0463d46937f6 nodeName:}" failed. No retries permitted until 2026-11-02 16:01:40.564393922 +0000 UTC m=+3907.002296080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls") pod "maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" (UID: "30631da0-b35f-4a13-8033-0463d46937f6") : secret "maas-api-e2e-authcfg-http-2afef9-serving-cert" not found
Nov 02 16:01:40.064874 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.064850    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.066885 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.066859    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.125862 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.125787    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v759\" (UniqueName: \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.570819 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.570775    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.573341 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.573311    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:40.772636 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:40.772596    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:41.152223 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:41.152171    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:01:41.154698 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:01:41.154662    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30631da0_b35f_4a13_8033_0463d46937f6.slice/crio-d6823bed05004ee7d821829d70927a2d689fbdc665c3ecc6b0f2d52e0d08922a WatchSource:0}: Error finding container d6823bed05004ee7d821829d70927a2d689fbdc665c3ecc6b0f2d52e0d08922a: Status 404 returned error can't find the container with id d6823bed05004ee7d821829d70927a2d689fbdc665c3ecc6b0f2d52e0d08922a
Nov 02 16:01:41.853732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:41.853691    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" event={"ID":"30631da0-b35f-4a13-8033-0463d46937f6","Type":"ContainerStarted","Data":"d6823bed05004ee7d821829d70927a2d689fbdc665c3ecc6b0f2d52e0d08922a"}
Nov 02 16:01:42.861525 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:42.861483    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" event={"ID":"30631da0-b35f-4a13-8033-0463d46937f6","Type":"ContainerStarted","Data":"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"}
Nov 02 16:01:42.861997 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:42.861638    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:42.981032 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:42.980971    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" podStartSLOduration=3.13360402 podStartE2EDuration="3.980954689s" totalImagesPullingTime="847.350669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:01:42.979124351 +0000 UTC m=+3909.417026493" watchObservedRunningTime="2026-11-02 16:01:42.980954689 +0000 UTC m=+3909.418856833"
Nov 02 16:01:48.894875 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:48.894777    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:01:49.844684 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:49.844650    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:01:49.844888 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:49.844750    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:01:57.943919 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.943882    4470 generic.go:356] "Generic (PLEG): container finished" podID="47566184-c99c-4402-9772-553987d16284" containerID="06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3" exitCode=0
Nov 02 16:01:57.943919 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.943923    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerDied","Data":"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"}
Nov 02 16:01:57.944466 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.944326    4470 scope.go:122] "RemoveContainer" containerID="06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"
Nov 02 16:01:57.945676 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.945644    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:57.945763 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.945720    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:57.945763 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:57.945754    4470 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 16:01:58.951684 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:58.951649    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerStarted","Data":"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"}
Nov 02 16:01:59.862289 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:59.862251    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:01:59.863493 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:59.863468    4470 patch_prober.go:29] interesting pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 02 16:01:59.863619 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:01:59.863512    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 02 16:02:00.862114 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:00.862078    4470 patch_prober.go:29] interesting pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 02 16:02:00.862556 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:00.862133    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 02 16:02:01.861739 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:01.861697    4470 patch_prober.go:29] interesting pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 02 16:02:01.861966 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:01.861755    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 02 16:02:02.800893 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:02.800847    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:02:02.862686 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:02.862646    4470 patch_prober.go:29] interesting pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 02 16:02:02.862870 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:02.862705    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 02 16:02:03.565019 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:02:03.564984    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:02:03.862403 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:03.862313    4470 patch_prober.go:29] interesting pod/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused" start-of-body=
Nov 02 16:02:03.862403 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:03.862384    4470 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.163:15021/healthz/ready\": dial tcp 10.133.0.163:15021: connect: connection refused"
Nov 02 16:02:04.867563 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:04.867530    4470 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:02:04.868533 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:02:04.868511    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:02:33.566217 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:02:33.566165    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:03:03.564778 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:03.564734    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:03:31.528593 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:31.528501    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:03:31.637420 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:31.637383    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:03:33.565396 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:33.565352    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:03:36.529354 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.529262    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" podUID="7d502d62-fd41-448c-a2a0-424d4a4f4278" containerName="payload-processing" containerID="cri-o://a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2" gracePeriod=25
Nov 02 16:03:36.638383 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.638310    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" podUID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" containerName="payload-pre-processing" containerID="cri-o://e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761" gracePeriod=25
Nov 02 16:03:36.783148 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.783074    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:03:36.845051 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.845014    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\") pod \"7d502d62-fd41-448c-a2a0-424d4a4f4278\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") "
Nov 02 16:03:36.845264 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.845121    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\" (UniqueName: \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\") pod \"7d502d62-fd41-448c-a2a0-424d4a4f4278\" (UID: \"7d502d62-fd41-448c-a2a0-424d4a4f4278\") "
Nov 02 16:03:36.845439 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.845414    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume" pod "7d502d62-fd41-448c-a2a0-424d4a4f4278" (UID: "7d502d62-fd41-448c-a2a0-424d4a4f4278"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:03:36.847334 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.847308    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64" pod "7d502d62-fd41-448c-a2a0-424d4a4f4278" (UID: "7d502d62-fd41-448c-a2a0-424d4a4f4278"). InnerVolumeSpecName "kube-api-access-jnk64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:03:36.888817 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.888791    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:03:36.946044 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.946006    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\" (UniqueName: \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\") pod \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") "
Nov 02 16:03:36.946239 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.946105    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\") pod \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\" (UID: \"c94bfe95-d49d-4942-9bb7-d81ba9dd2215\") "
Nov 02 16:03:36.946529 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.946508    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d502d62-fd41-448c-a2a0-424d4a4f4278-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:36.946573 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.946536    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnk64\" (UniqueName: \"kubernetes.io/projected/7d502d62-fd41-448c-a2a0-424d4a4f4278-kube-api-access-jnk64\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:36.946573 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.946549    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume" pod "c94bfe95-d49d-4942-9bb7-d81ba9dd2215" (UID: "c94bfe95-d49d-4942-9bb7-d81ba9dd2215"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:03:36.948124 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:36.948102    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj" pod "c94bfe95-d49d-4942-9bb7-d81ba9dd2215" (UID: "c94bfe95-d49d-4942-9bb7-d81ba9dd2215"). InnerVolumeSpecName "kube-api-access-jq9qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:03:37.047823 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.047737    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq9qj\" (UniqueName: \"kubernetes.io/projected/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-kube-api-access-jq9qj\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:37.047823 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.047765    4470 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c94bfe95-d49d-4942-9bb7-d81ba9dd2215-plugins-config-volume\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:37.465012 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.464979    4470 generic.go:356] "Generic (PLEG): container finished" podID="7d502d62-fd41-448c-a2a0-424d4a4f4278" containerID="a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2" exitCode=0
Nov 02 16:03:37.465223 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.465026    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"
Nov 02 16:03:37.465223 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.465063    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" event={"ID":"7d502d62-fd41-448c-a2a0-424d4a4f4278","Type":"ContainerDied","Data":"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"}
Nov 02 16:03:37.465223 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.465129    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x" event={"ID":"7d502d62-fd41-448c-a2a0-424d4a4f4278","Type":"ContainerDied","Data":"c2e2cae6a40cfaa2147454425d446c11915be39840219db81035f31a01f191fe"}
Nov 02 16:03:37.465223 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.465161    4470 scope.go:122] "RemoveContainer" containerID="a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"
Nov 02 16:03:37.466382 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.466360    4470 generic.go:356] "Generic (PLEG): container finished" podID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" containerID="e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761" exitCode=0
Nov 02 16:03:37.466471 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.466400    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" event={"ID":"c94bfe95-d49d-4942-9bb7-d81ba9dd2215","Type":"ContainerDied","Data":"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"}
Nov 02 16:03:37.466471 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.466434    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49" event={"ID":"c94bfe95-d49d-4942-9bb7-d81ba9dd2215","Type":"ContainerDied","Data":"7f957e0025a79df890bb87218ab82ae68ac46b8562b3dba3a0644302f480cd3f"}
Nov 02 16:03:37.466471 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.466402    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"
Nov 02 16:03:37.474920 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.474896    4470 scope.go:122] "RemoveContainer" containerID="a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"
Nov 02 16:03:37.475315 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:37.475293    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2\": container with ID starting with a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2 not found: ID does not exist" containerID="a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"
Nov 02 16:03:37.475374 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.475324    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2"} err="failed to get container status \"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2\": rpc error: code = NotFound desc = could not find container \"a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2\": container with ID starting with a93dfce057de1a1019efc9fcbcd5b07ad86edf0658f400983a18a6bf10011bc2 not found: ID does not exist"
Nov 02 16:03:37.475374 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.475340    4470 scope.go:122] "RemoveContainer" containerID="e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"
Nov 02 16:03:37.485820 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.485778    4470 scope.go:122] "RemoveContainer" containerID="e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"
Nov 02 16:03:37.486243 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:37.486225    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761\": container with ID starting with e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761 not found: ID does not exist" containerID="e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"
Nov 02 16:03:37.486322 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.486254    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761"} err="failed to get container status \"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761\": rpc error: code = NotFound desc = could not find container \"e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761\": container with ID starting with e0787e783940e1f571cc07b1dcc7ce03ae9594643feaa2945949032b3a068761 not found: ID does not exist"
Nov 02 16:03:37.629794 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.629757    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:03:37.679020 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.678978    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-2afef9-7b86cc8865-5fz2x"]
Nov 02 16:03:37.705739 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.705707    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d502d62-fd41-448c-a2a0-424d4a4f4278" path="/var/lib/kubelet/pods/7d502d62-fd41-448c-a2a0-424d4a4f4278/volumes"
Nov 02 16:03:37.743340 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.743256    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:03:37.747333 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:37.747303    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-2afef9-54cd45b4c-sxd49"]
Nov 02 16:03:39.706827 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:39.706784    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" path="/var/lib/kubelet/pods/c94bfe95-d49d-4942-9bb7-d81ba9dd2215/volumes"
Nov 02 16:03:40.853560 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:40.853522    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:03:40.853988 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:40.853731    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" podUID="30631da0-b35f-4a13-8033-0463d46937f6" containerName="maas-api" containerID="cri-o://8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098" gracePeriod=30
Nov 02 16:03:41.124977 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.124946    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:03:41.192148 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.192113    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\") pod \"30631da0-b35f-4a13-8033-0463d46937f6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") "
Nov 02 16:03:41.192365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.192238    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\" (UniqueName: \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\") pod \"30631da0-b35f-4a13-8033-0463d46937f6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") "
Nov 02 16:03:41.192365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.192274    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\") pod \"30631da0-b35f-4a13-8033-0463d46937f6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") "
Nov 02 16:03:41.192365 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.192322    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") pod \"30631da0-b35f-4a13-8033-0463d46937f6\" (UID: \"30631da0-b35f-4a13-8033-0463d46937f6\") "
Nov 02 16:03:41.192688 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.192657    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca" pod "30631da0-b35f-4a13-8033-0463d46937f6" (UID: "30631da0-b35f-4a13-8033-0463d46937f6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:03:41.194398 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.194357    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls" pod "30631da0-b35f-4a13-8033-0463d46937f6" (UID: "30631da0-b35f-4a13-8033-0463d46937f6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:03:41.194506 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.194417    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs" pod "30631da0-b35f-4a13-8033-0463d46937f6" (UID: "30631da0-b35f-4a13-8033-0463d46937f6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 16:03:41.194506 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.194464    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759" pod "30631da0-b35f-4a13-8033-0463d46937f6" (UID: "30631da0-b35f-4a13-8033-0463d46937f6"). InnerVolumeSpecName "kube-api-access-4v759". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:03:41.293994 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.293952    4470 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-metrics-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:41.293994 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.293988    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v759\" (UniqueName: \"kubernetes.io/projected/30631da0-b35f-4a13-8033-0463d46937f6-kube-api-access-4v759\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:41.293994 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.294000    4470 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30631da0-b35f-4a13-8033-0463d46937f6-openshift-service-ca\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:41.294282 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.294008    4470 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30631da0-b35f-4a13-8033-0463d46937f6-maas-api-tls\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:41.487949 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.487915    4470 generic.go:356] "Generic (PLEG): container finished" podID="30631da0-b35f-4a13-8033-0463d46937f6" containerID="8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098" exitCode=0
Nov 02 16:03:41.488146 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.487960    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"
Nov 02 16:03:41.488146 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.487998    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" event={"ID":"30631da0-b35f-4a13-8033-0463d46937f6","Type":"ContainerDied","Data":"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"}
Nov 02 16:03:41.488146 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.488062    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6" event={"ID":"30631da0-b35f-4a13-8033-0463d46937f6","Type":"ContainerDied","Data":"d6823bed05004ee7d821829d70927a2d689fbdc665c3ecc6b0f2d52e0d08922a"}
Nov 02 16:03:41.488146 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.488086    4470 scope.go:122] "RemoveContainer" containerID="8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"
Nov 02 16:03:41.498146 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.498127    4470 scope.go:122] "RemoveContainer" containerID="8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"
Nov 02 16:03:41.498437 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:41.498420    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098\": container with ID starting with 8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098 not found: ID does not exist" containerID="8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"
Nov 02 16:03:41.498496 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.498450    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098"} err="failed to get container status \"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098\": rpc error: code = NotFound desc = could not find container \"8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098\": container with ID starting with 8a9c340ac2e9089776758c86a043d972a88eb1ed689561c0f983444bbb2f1098 not found: ID does not exist"
Nov 02 16:03:41.652926 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.652888    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:03:41.705516 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:41.705489    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-2afef9-5ccdc474f7-nvjh6"]
Nov 02 16:03:43.705616 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:43.705581    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30631da0-b35f-4a13-8033-0463d46937f6" path="/var/lib/kubelet/pods/30631da0-b35f-4a13-8033-0463d46937f6/volumes"
Nov 02 16:03:49.930322 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:49.930276    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:03:49.930791 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:49.930583    4470 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy" containerID="cri-o://61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5" gracePeriod=30
Nov 02 16:03:55.184862 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.184832    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:03:55.347495 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347405    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347495 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347446    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347518    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347565    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347600    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347629    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347666    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.347732 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347712    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.348085 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347750    4470 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\") pod \"47566184-c99c-4402-9772-553987d16284\" (UID: \"47566184-c99c-4402-9772-553987d16284\") "
Nov 02 16:03:55.348085 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347894    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:03:55.348085 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347909    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:03:55.348085 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.347971    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:03:55.348339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.348134    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:03:55.348339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.348146    4470 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-data\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.348339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.348177    4470 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-credential-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.348339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.348141    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 16:03:55.348339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.348210    4470 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-certs\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.349809 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.349784    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 16:03:55.349926 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.349884    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "kube-api-access-j2pbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:03:55.350054 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.349910    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 16:03:55.350054 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.349965    4470 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo" pod "47566184-c99c-4402-9772-553987d16284" (UID: "47566184-c99c-4402-9772-553987d16284"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 16:03:55.449438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449396    4470 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-istio-token\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.449438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449430    4470 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/47566184-c99c-4402-9772-553987d16284-istiod-ca-cert\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.449438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449440    4470 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2pbd\" (UniqueName: \"kubernetes.io/projected/47566184-c99c-4402-9772-553987d16284-kube-api-access-j2pbd\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.449438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449450    4470 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-workload-socket\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.449750 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449459    4470 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/47566184-c99c-4402-9772-553987d16284-istio-podinfo\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.449750 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.449469    4470 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/47566184-c99c-4402-9772-553987d16284-istio-envoy\") on node \"ip-10-0-139-197.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 16:03:55.560641 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.560608    4470 generic.go:356] "Generic (PLEG): container finished" podID="47566184-c99c-4402-9772-553987d16284" containerID="61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5" exitCode=0
Nov 02 16:03:55.560828 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.560658    4470 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"
Nov 02 16:03:55.560828 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.560666    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerDied","Data":"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"}
Nov 02 16:03:55.560828 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.560708    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7" event={"ID":"47566184-c99c-4402-9772-553987d16284","Type":"ContainerDied","Data":"b8fbc7ba0d550c083ee9a1c334f7c4e3326749f938763eec7b8a7c7d17a34f85"}
Nov 02 16:03:55.560828 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.560730    4470 scope.go:122] "RemoveContainer" containerID="61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"
Nov 02 16:03:55.570499 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.570480    4470 scope.go:122] "RemoveContainer" containerID="06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"
Nov 02 16:03:55.578728 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.578706    4470 scope.go:122] "RemoveContainer" containerID="61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"
Nov 02 16:03:55.578989 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:55.578969    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5\": container with ID starting with 61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5 not found: ID does not exist" containerID="61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"
Nov 02 16:03:55.579069 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.579003    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5"} err="failed to get container status \"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5\": rpc error: code = NotFound desc = could not find container \"61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5\": container with ID starting with 61f2fbf01cbc57bc54f09d8c75427a3f7ee8c0cb127010717796c6d329ef3ea5 not found: ID does not exist"
Nov 02 16:03:55.579069 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.579027    4470 scope.go:122] "RemoveContainer" containerID="06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"
Nov 02 16:03:55.579294 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:03:55.579273    4470 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3\": container with ID starting with 06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3 not found: ID does not exist" containerID="06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"
Nov 02 16:03:55.579346 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.579301    4470 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3"} err="failed to get container status \"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3\": rpc error: code = NotFound desc = could not find container \"06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3\": container with ID starting with 06b05fe366fcecbf460757f5b62598d6d0b3d3edc0c87a3e56b82ccc70576da3 not found: ID does not exist"
Nov 02 16:03:55.733307 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.733269    4470 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:03:55.737195 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:55.737164    4470 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-2afef9-openshift-default-5cdcbb8949-7r4h7"]
Nov 02 16:03:57.705252 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:03:57.705210    4470 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="47566184-c99c-4402-9772-553987d16284" path="/var/lib/kubelet/pods/47566184-c99c-4402-9772-553987d16284/volumes"
Nov 02 16:04:03.564849 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:04:03.564810    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:04:12.087212 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:04:12.086315    4470 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75fd9f5d47-5j55g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 16:04:12.087212 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:04:12.086406    4470 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75fd9f5d47-5j55g" podUID="9e86ef4a-26ce-435d-a63f-f951e10ee840" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 16:04:27.802357 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:04:27.802311    4470 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal" podUID="6e7f87aa5e9c9e37d740f6ee1d873224" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Nov 02 16:04:33.564825 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:04:33.564785    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:05:03.564601 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:05:03.564560    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:05:14.295600 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:14.295559    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f9b4d7758-wc49n_34d7b14a-ff01-4807-99c4-14c4f9df1511/manager/0.log"
Nov 02 16:05:15.676118 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:15.676081    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:05:16.313575 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:16.313539    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-62ftm_f5b0ee7b-b9b4-4029-9cad-532b3c334a5b/manager/0.log"
Nov 02 16:05:16.555307 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:16.555246    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4v492_87736579-2a3c-48d5-b5fa-6c39c2726194/proxy/0.log"
Nov 02 16:05:17.598491 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:17.598453    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d95f6c575-q7vqg_ea565e18-23df-4e7a-8443-df50f4767c56/maas-api/0.log"
Nov 02 16:05:18.481247 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:18.481219    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2srjn_6256836a-d15d-4a91-a040-3807456115b3/postgres/0.log"
Nov 02 16:05:24.157917 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:24.157881    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-5j55g_9e86ef4a-26ce-435d-a63f-f951e10ee840/istio-proxy/0.log"
Nov 02 16:05:24.828526 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:24.828494    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-cw6h6_0169e1af-5286-420d-8fe8-96f88517b525/istio-proxy/0.log"
Nov 02 16:05:25.049093 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:25.049059    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-fxb56_acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396/istio-proxy/0.log"
Nov 02 16:05:25.770582 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:25.770553    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-vdl45_92bb99de-1e3b-4f61-9cac-13b3c9496758/istio-proxy/0.log"
Nov 02 16:05:27.595607 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:27.595576    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7_1da922da-c4c3-4c35-9edb-45f6bd072094/main/0.log"
Nov 02 16:05:27.868601 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:27.868524    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-rqxwp_0aa36d7b-d894-497a-8f79-3f68a36b0f88/main/0.log"
Nov 02 16:05:28.353702 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:28.353668    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-mdbl2_f2ace023-874e-4aa4-a6c7-d91d3f83fb1c/main/0.log"
Nov 02 16:05:28.568536 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:28.568501    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2_c00969f6-55d8-46d4-b90e-6391709f1798/main/0.log"
Nov 02 16:05:28.802081 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:28.802053    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-fxkf2_b51259a2-7ada-4385-82b0-3586cfdb59ba/main/0.log"
Nov 02 16:05:33.568832 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:05:33.568789    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:05:39.417058 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:39.417021    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6f9b4d7758-wc49n_34d7b14a-ff01-4807-99c4-14c4f9df1511/manager/0.log"
Nov 02 16:05:40.800823 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:40.800782    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:05:41.355044 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:41.355011    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-62ftm_f5b0ee7b-b9b4-4029-9cad-532b3c334a5b/manager/0.log"
Nov 02 16:05:41.615727 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:41.615640    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-4v492_87736579-2a3c-48d5-b5fa-6c39c2726194/proxy/0.log"
Nov 02 16:05:42.436089 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:42.436055    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-d95f6c575-q7vqg_ea565e18-23df-4e7a-8443-df50f4767c56/maas-api/0.log"
Nov 02 16:05:43.365669 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:43.365637    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-2srjn_6256836a-d15d-4a91-a040-3807456115b3/postgres/0.log"
Nov 02 16:05:48.618837 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:48.618804    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-5j55g_9e86ef4a-26ce-435d-a63f-f951e10ee840/istio-proxy/0.log"
Nov 02 16:05:49.392433 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:49.392344    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-cw6h6_0169e1af-5286-420d-8fe8-96f88517b525/istio-proxy/0.log"
Nov 02 16:05:49.624509 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:49.624469    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-fxb56_acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396/istio-proxy/0.log"
Nov 02 16:05:50.360734 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:50.360704    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-vdl45_92bb99de-1e3b-4f61-9cac-13b3c9496758/istio-proxy/0.log"
Nov 02 16:05:51.980264 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:51.980232    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-fblm7_1da922da-c4c3-4c35-9edb-45f6bd072094/main/0.log"
Nov 02 16:05:52.257499 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:52.257404    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-rqxwp_0aa36d7b-d894-497a-8f79-3f68a36b0f88/main/0.log"
Nov 02 16:05:52.763171 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:52.763139    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-mdbl2_f2ace023-874e-4aa4-a6c7-d91d3f83fb1c/main/0.log"
Nov 02 16:05:53.005652 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:53.005615    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8c5cg2_c00969f6-55d8-46d4-b90e-6391709f1798/main/0.log"
Nov 02 16:05:53.241203 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:05:53.241166    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-fxkf2_b51259a2-7ada-4385-82b0-3586cfdb59ba/main/0.log"
Nov 02 16:06:03.564337 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:06:03.564300    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:06:20.230589 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:06:20.230514    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:06:20.812377 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:06:20.812343    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:06:33.565878 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:06:33.565835    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:06:34.240778 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:06:34.240750    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 16:06:34.291998 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:06:34.291972    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kndjb_8e573dd1-b81f-4134-9454-739323385c26/ovn-acl-logging/0.log"
Nov 02 16:07:03.565116 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:07:03.565076    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:07:07.561111 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:07.561072    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:07:08.143567 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:08.143536    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-76976749fb-gn8d9_5ef0279a-f41b-46ac-98e4-aa329ff8395f/manager/0.log"
Nov 02 16:07:09.530438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530390    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjv5q/must-gather-tpz7t"]
Nov 02 16:07:09.530870 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530827    4470 cpu_manager.go:414] "containerMap: removing container" podUID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" containerName="payload-pre-processing"
Nov 02 16:07:09.530870 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530838    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" containerName="payload-pre-processing"
Nov 02 16:07:09.530870 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530857    4470 cpu_manager.go:414] "containerMap: removing container" podUID="30631da0-b35f-4a13-8033-0463d46937f6" containerName="maas-api"
Nov 02 16:07:09.530870 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530863    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30631da0-b35f-4a13-8033-0463d46937f6" containerName="maas-api"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530873    4470 cpu_manager.go:414] "containerMap: removing container" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530878    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530901    4470 cpu_manager.go:414] "containerMap: removing container" podUID="7d502d62-fd41-448c-a2a0-424d4a4f4278" containerName="payload-processing"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530908    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d502d62-fd41-448c-a2a0-424d4a4f4278" containerName="payload-processing"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530971    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530978    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d502d62-fd41-448c-a2a0-424d4a4f4278" containerName="payload-processing"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530987    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="c94bfe95-d49d-4942-9bb7-d81ba9dd2215" containerName="payload-pre-processing"
Nov 02 16:07:09.531011 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.530997    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="30631da0-b35f-4a13-8033-0463d46937f6" containerName="maas-api"
Nov 02 16:07:09.531291 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.531082    4470 cpu_manager.go:414] "containerMap: removing container" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.531291 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.531087    4470 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.531291 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.531151    4470 memory_manager.go:365] "RemoveStaleState removing state" podUID="47566184-c99c-4402-9772-553987d16284" containerName="istio-proxy"
Nov 02 16:07:09.534171 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.534154    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.565753 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.565715    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0536d3aa-e3b4-4901-b3a2-0264a08a292a-must-gather-output\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.565937 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.565842    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9866t\" (UniqueName: \"kubernetes.io/projected/0536d3aa-e3b4-4901-b3a2-0264a08a292a-kube-api-access-9866t\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.587671 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.587637    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjv5q\"/\"kube-root-ca.crt\""
Nov 02 16:07:09.637023 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.636984    4470 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gjv5q\"/\"openshift-service-ca.crt\""
Nov 02 16:07:09.637023 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.637004    4470 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gjv5q\"/\"default-dockercfg-2qgwv\""
Nov 02 16:07:09.639027 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.638997    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjv5q/must-gather-tpz7t"]
Nov 02 16:07:09.666688 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.666644    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0536d3aa-e3b4-4901-b3a2-0264a08a292a-must-gather-output\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.666872 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.666760    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9866t\" (UniqueName: \"kubernetes.io/projected/0536d3aa-e3b4-4901-b3a2-0264a08a292a-kube-api-access-9866t\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.667043 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.667021    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0536d3aa-e3b4-4901-b3a2-0264a08a292a-must-gather-output\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.728247 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.728211    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9866t\" (UniqueName: \"kubernetes.io/projected/0536d3aa-e3b4-4901-b3a2-0264a08a292a-kube-api-access-9866t\") pod \"must-gather-tpz7t\" (UID: \"0536d3aa-e3b4-4901-b3a2-0264a08a292a\") " pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:09.848374 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:09.848283    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjv5q/must-gather-tpz7t"
Nov 02 16:07:10.233346 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:10.233321    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjv5q/must-gather-tpz7t"]
Nov 02 16:07:10.235894 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:07:10.235860    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0536d3aa_e3b4_4901_b3a2_0264a08a292a.slice/crio-66ee2060e180daf72f67b0596489bc8d1d184d0c9922ff6bb437023860a66d2f WatchSource:0}: Error finding container 66ee2060e180daf72f67b0596489bc8d1d184d0c9922ff6bb437023860a66d2f: Status 404 returned error can't find the container with id 66ee2060e180daf72f67b0596489bc8d1d184d0c9922ff6bb437023860a66d2f
Nov 02 16:07:10.237941 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:10.237921    4470 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 16:07:10.612374 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:10.612280    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjv5q/must-gather-tpz7t" event={"ID":"0536d3aa-e3b4-4901-b3a2-0264a08a292a","Type":"ContainerStarted","Data":"66ee2060e180daf72f67b0596489bc8d1d184d0c9922ff6bb437023860a66d2f"}
Nov 02 16:07:11.622359 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:11.622314    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjv5q/must-gather-tpz7t" event={"ID":"0536d3aa-e3b4-4901-b3a2-0264a08a292a","Type":"ContainerStarted","Data":"df00db015e73418998a5b1da6fecd6df16723057ea385d3ebbd9d8ae6d2900cd"}
Nov 02 16:07:12.631267 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:12.631224    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjv5q/must-gather-tpz7t" event={"ID":"0536d3aa-e3b4-4901-b3a2-0264a08a292a","Type":"ContainerStarted","Data":"1b1455ede5ac03bf1f1716ce6231fb8909e8abc3a6248f520b5ddefd5d926f33"}
Nov 02 16:07:12.749293 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:12.749236    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-gjv5q/must-gather-tpz7t" podStartSLOduration=2.598293275 podStartE2EDuration="3.749220356s" totalImagesPullingTime="1.150927081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 16:07:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:07:12.747252085 +0000 UTC m=+4239.185154226" watchObservedRunningTime="2026-11-02 16:07:12.749220356 +0000 UTC m=+4239.187122497"
Nov 02 16:07:18.328215 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:18.326257    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d7qbm_fb9bfe09-6645-4ba6-bdbb-62a02364fbb1/global-pull-secret-syncer/0.log"
Nov 02 16:07:22.633287 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:22.633249    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vsttp_ba79c206-9b29-4cc1-a3cd-30dc9f2c60e3/konnectivity-agent/0.log"
Nov 02 16:07:23.855968 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:23.855923    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-197.us-west-1.compute.internal_6e7f87aa5e9c9e37d740f6ee1d873224/haproxy/0.log"
Nov 02 16:07:33.293816 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:33.293756    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/alertmanager/0.log"
Nov 02 16:07:33.566606 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:07:33.566511    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:07:33.727434 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:33.727398    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/config-reloader/0.log"
Nov 02 16:07:34.147403 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:34.147371    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/kube-rbac-proxy-web/0.log"
Nov 02 16:07:34.569964 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:34.569927    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/kube-rbac-proxy/0.log"
Nov 02 16:07:34.986018 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:34.985984    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/kube-rbac-proxy-metric/0.log"
Nov 02 16:07:35.403091 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:35.403004    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/prom-label-proxy/0.log"
Nov 02 16:07:35.822377 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:35.822343    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c2a70169-94fd-4dc9-b3e2-6483373fe14c/init-config-reloader/0.log"
Nov 02 16:07:38.015929 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:38.015883    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-bc66565f-8htjh_6c2d4dc1-08a3-4e50-be9a-182def88ba72/metrics-server/0.log"
Nov 02 16:07:38.434130 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:38.434098    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-w4bkt_2ebe193b-b3c8-4197-b516-506c88b0dc33/monitoring-plugin/0.log"
Nov 02 16:07:38.856339 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:38.856261    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvwkz_6f1f75d6-4610-427e-a5a1-0396c925b1ba/node-exporter/0.log"
Nov 02 16:07:39.269157 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:39.269115    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvwkz_6f1f75d6-4610-427e-a5a1-0396c925b1ba/kube-rbac-proxy/0.log"
Nov 02 16:07:39.694978 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:39.684282    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bvwkz_6f1f75d6-4610-427e-a5a1-0396c925b1ba/init-textfile/0.log"
Nov 02 16:07:46.812504 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:46.812375    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-82lbh_d0c5373c-f6e3-47e5-adfc-7bb0496d9b37/prometheus-operator/0.log"
Nov 02 16:07:47.221472 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:47.221437    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-82lbh_d0c5373c-f6e3-47e5-adfc-7bb0496d9b37/kube-rbac-proxy/0.log"
Nov 02 16:07:47.640511 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:47.640416    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-zvdzm_06b352aa-9fb8-4e3e-a4de-944438706944/prometheus-operator-admission-webhook/0.log"
Nov 02 16:07:48.247262 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.247219    4470 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"]
Nov 02 16:07:48.252454 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.252426    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.343640 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.343600    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"]
Nov 02 16:07:48.381174 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.381140    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92494\" (UniqueName: \"kubernetes.io/projected/30ce512e-b50d-48bc-b4a2-cc20e35435fd-kube-api-access-92494\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.381378 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.381275    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-proc\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.381378 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.381315    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-lib-modules\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.381378 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.381343    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-sys\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.381505 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.381465    4470 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-podres\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482573 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482536    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-lib-modules\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482592    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-sys\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482664    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-podres\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482695    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-lib-modules\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482722    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92494\" (UniqueName: \"kubernetes.io/projected/30ce512e-b50d-48bc-b4a2-cc20e35435fd-kube-api-access-92494\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482731    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-sys\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.482810 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482764    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-podres\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.483127 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482818    4470 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-proc\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.483127 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.482890    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/30ce512e-b50d-48bc-b4a2-cc20e35435fd-proc\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.543760 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.543684    4470 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92494\" (UniqueName: \"kubernetes.io/projected/30ce512e-b50d-48bc-b4a2-cc20e35435fd-kube-api-access-92494\") pod \"perf-node-gather-daemonset-46t5v\" (UID: \"30ce512e-b50d-48bc-b4a2-cc20e35435fd\") " pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.566624 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.566584    4470 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:48.946243 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:48.946164    4470 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"]
Nov 02 16:07:48.948762 ip-10-0-139-197 kubenswrapper[4470]: W1102 16:07:48.948737    4470 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod30ce512e_b50d_48bc_b4a2_cc20e35435fd.slice/crio-bfadb962f2c0a54afd3e54456050858029f6344d2423a2eeed875df7616fcd06 WatchSource:0}: Error finding container bfadb962f2c0a54afd3e54456050858029f6344d2423a2eeed875df7616fcd06: Status 404 returned error can't find the container with id bfadb962f2c0a54afd3e54456050858029f6344d2423a2eeed875df7616fcd06
Nov 02 16:07:49.314594 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.314499    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/thanos-query/0.log"
Nov 02 16:07:49.732392 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.732362    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/kube-rbac-proxy-web/0.log"
Nov 02 16:07:49.858847 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.858806    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v" event={"ID":"30ce512e-b50d-48bc-b4a2-cc20e35435fd","Type":"ContainerStarted","Data":"9292e3f0a306b103ab2918c94ed291f3a27f3a4f09453dccb8b8c371eb7f0d93"}
Nov 02 16:07:49.859035 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.858873    4470 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v" event={"ID":"30ce512e-b50d-48bc-b4a2-cc20e35435fd","Type":"ContainerStarted","Data":"bfadb962f2c0a54afd3e54456050858029f6344d2423a2eeed875df7616fcd06"}
Nov 02 16:07:49.859035 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.859014    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:49.974738 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:49.974550    4470 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v" podStartSLOduration=1.97453272 podStartE2EDuration="1.97453272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 16:07:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 16:07:49.973930898 +0000 UTC m=+4276.411833040" watchObservedRunningTime="2026-11-02 16:07:49.97453272 +0000 UTC m=+4276.412434862"
Nov 02 16:07:50.155393 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:50.155312    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/kube-rbac-proxy/0.log"
Nov 02 16:07:50.572555 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:50.572520    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/prom-label-proxy/0.log"
Nov 02 16:07:50.987438 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:50.987376    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/kube-rbac-proxy-rules/0.log"
Nov 02 16:07:51.403165 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:51.403094    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56964c66d-wjrwv_629f5487-7949-49df-8794-8bf26067bea3/kube-rbac-proxy-metrics/0.log"
Nov 02 16:07:55.873156 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:55.873128    4470 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gjv5q/perf-node-gather-daemonset-46t5v"
Nov 02 16:07:59.333623 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:07:59.333592    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d9fc999b6-686kf_8c809cf8-9aaa-4cff-bc52-712a3de6f673/console/0.log"
Nov 02 16:08:03.565307 ip-10-0-139-197 kubenswrapper[4470]: E1102 16:08:03.565264    4470 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fjzx9" containerName="extractor"
Nov 02 16:08:07.962753 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:07.962723    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5t56l_072126f8-4f44-46b1-a1ca-221a57d7feb3/dns/0.log"
Nov 02 16:08:08.374631 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:08.374551    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5t56l_072126f8-4f44-46b1-a1ca-221a57d7feb3/kube-rbac-proxy/0.log"
Nov 02 16:08:11.309857 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:11.309828    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xrvk7_ed89b459-a1e3-4cb7-a05c-1e79ba406239/dns-node-resolver/0.log"
Nov 02 16:08:15.483081 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:15.483045    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ldxkl_bd0c7197-ef21-4d95-94bc-133fbd54c166/node-ca/0.log"
Nov 02 16:08:23.088920 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:23.088884    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-5j55g_9e86ef4a-26ce-435d-a63f-f951e10ee840/istio-proxy/0.log"
Nov 02 16:08:24.355182 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:24.355143    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-cw6h6_0169e1af-5286-420d-8fe8-96f88517b525/istio-proxy/0.log"
Nov 02 16:08:24.785023 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:24.784984    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-fxb56_acf6ef44-d2ec-4e80-96bd-d3aa0cfc1396/istio-proxy/0.log"
Nov 02 16:08:26.051880 ip-10-0-139-197 kubenswrapper[4470]: I1102 16:08:26.051838    4470 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75fd9f5d47-vdl45_92bb99de-1e3b-4f61-9cac-13b3c9496758/istio-proxy/0.log"